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Minimax  problems  are  fundamented  to  nonlinear  programming,  because 
f  the  way  constraints  can  be  represented  using  Lagrange  multipliers, 
etter  ways  of  solving  minimax  problems  would  lead  thus  lead  to  break- 
hroughs  in  sovling  most  other  problems  of  optimization.  This 
issertation  opens  a  new  avenue  to  the  study  of  minimax  problems  by 
eveloping  a  theory  of  dual  operations  on  saddle-functions  convex-concav< 
unctions  parallel  to,  that  already  known  for  (purely)  convex  functions, 
esults  are  thereby  obtained  concerning  minimax  problems  which  are 
ual  to  each  other.  It  is  expected  that  these  results  will  find 
omputatlonal  applications  analogous  to  those  already  acclaimed  in  the 
onvex  case,  for  instance  in  decomposition  of  large-scale  problems. 


DD  .'“‘..1473 


CONIfM*! 


Acknowledgments 

1  * 

Preface 

1  '4 

Inirxcjct  loti 

1 

■iO 

f’rel  iminarles 

'1 

51 

A  General  Theorem 

20 

52 

Two  Dual  Operations 

27 

53: 

Soar per  Results 

36 

54 : 

Addition  enc  Mini max  convolution 

43 

«£; 

The  Postal  Conjugacy  Operation 

<■>0 

56: 

Generalized  Saddle  Program?. 

% 

i  7 : 

Ordinary  Sadd’e  Programs  ami  Lagrange  Multipliers 

87 

'38: 

Saddle  Programs  of  fenchel  Type 

99 

Appendix:  Polyhedral  Ref i nen-er. ts 

105 

References 

109 

Approved  for  public  release; 
distribution  uulloitod. 


Acknowledgments 


To  Professor  A.  Tyrrell  Rockefeller,  my  thesis  edvlsor,  I  wish  to  ex¬ 
press  n\y  gretltude  for  Introducing  me  to  convex  anelysls  and  for  suggestlrg 
the  problem  which  led  to  this  thesis.  Deep  epprecletlon  also  goes  to 
my  wife,  for  her  steadfast  encouragement.  I  owe  special  thanks  to  the  Air 
Force  Office  of  Scientific  Research  for  Its  financial  support  under  grant 
no.  4£AF0SR-71"L994and  to  the  National  Defense  Education  Act  for  a  generous 
fellowship  at  the  University  of  Washington. 


/ 


\ 


Preface 


Due  to  the  frequency  with  which  results  from  Rockafellti  [44.1  are  cited 
throughout  the  thesis,  a  special  abbreviation  is  used.  Namely,  the  number  of 
toe  result  being  cited  Is  given  enclosed  In,  parentheses.  For  example.  Theorem 
?38  Is  cited  as  (23.8),  Corollary  6.3.1  as  (6.3.1),  and  so  forth. 

Throughout  the  thesis  expressions  sometimes  appear  which  involve  taking 
the  supremum  or  Inflmum  of  an  empty  set  of  numbers.  Whenever  these  occur 
they  are  to  be  Interpreted  using  the  conventions  sup  d  *  -**  and  inf  0  *  +*. 

The  common  abbreviation  "Iff"  Is  used  for  the  phrase  "if,  and  only  If.' 

Finally,  for  si  the  reader  need  only  read  §0  up  to  lemma  0.3.  The  rest 
of  the  thesis  draws  on  all  of  SO. 


Introduction 


Mlnlmax  theory  may  be  said  to  have  originated  In  1028  with  von  Neumann's 
mlnlmax  theorem  for  matrix  games  [34],  Various  proofs  and  generalizations 
of  this  theorem  have  been  given  by  many  authors,  Including  VI lie  [59],  Kaku- 
tanl  [2f  j,  Wald  [60],  Shlffman  [50],  Fan  [18,  19],  Kncser  [27],  Gllcksberg 
[24],  Nika  Ido  [35],  Berge  [4],  Sion  [51],  Ghoulla-Kourl  [23],  Moreau  [33], 
and  Rockafellar  [39,  40]. 

Much  of  the  early  work  In  mlnlmax  theory  was  done  In  connection  with 
game  theory.  However  In  about  1950  two  equivalences  were  established  which 
made  it  apparent  that  mlnlmax  theory  had  much  relevance  for  mathematical  pro¬ 
gramming.  One  of  these  equivalences  was  that  between  matrix  games  and  dual 
pairs  of  linear  programs  (see  Dantzlg  [12],  Gale-Kuhrv-Tucker  [22],  and  Charnes 
[8]}..  The  other  equivalence  was -that  between  convex  programs  and  Lagranglan 
saddle-point  problems  (see  Kuhn-Tucker  [28],  Slater  [52],  and  extensions 
given  by  HurwIoHJzawa  in  [2]).  Various  authors.  Including  Stoer  [53,  54], 
Mangasarlan-Ponsteln  [31],  and  Dantzlg-Elsenberg-Cottle  [13],  later  derived 
duality  results  for  constrained  maximization  and  minimization  problems  by 
means  of  mlnlmax  theorems. 

In  [39]  Rockefeller  defined  a  conjugacy  correspondence  among  saddle- 
functions  parallel  to  that  of  Fenchel  [20]  for  convex  functions.  This  corvos- 
pondence  was  used  In  [43]  to  represent  (In  finitely  many  different  ways)  a 
certain  dual  pair  of  convex  programs  as  a  dual  pair  of  mlnlmax  problems.  At 
a  later  date  Tynjanskll  [57]  Independently  defined  the  conjugacy  correspon¬ 
dence  for  a  more  restrictive  class  of  saddle-functions.  He  used  It  to  assoc¬ 
iate  with  a  given  concave-convex  game  another  game  of  the  same  type,  and 
showed  how  solving  such  a  pair  of  "dual  games"  Is  equivalent  to  solving  a 
related  pair  of  convex  programs.  Also,  papers  of  Moreau  [33]  and  Ioffe- 
Tikhomirov  C2S]  contain  Implicit  results  concerning  the  conjugacy 
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correspondence  among  saddle-f unctions. 

The  relevance  of  mlnlmax  theory  to  mathematical  uconomics  has  long  been 
recognized,  dating  back  to  the  beginnings  of  game  theory.  More  recently, 
minimax  theory  has  been  useful  In  the  calculus  of  variations  and  optimal  con¬ 
trol  theory  (e.g.  Rockafellar  [47,  48]).  It  also  plays  a  role  In  differen¬ 
tial  games  (e.g.  Sakawa  [493;  see  also  the  survey  article  by  Berkovltz  [5]). 

Related  to  minima*  problems  are  max-min  problems,  i.e.  two-stage  prob¬ 
lems  of  the  form  max(min  f(x,y)).  These  have  been  studied  by  Pchenlchnyi 

x  y 

[36],  Oanskln  [10],  and  Bram  [6].  Such  problems  correspond  to  'half"  a 
saddle-point  problem  and  arise  from  such  practical  considerations  as  two- 
stage  resource  allocation. 

The  preceding  references  deal  prlmari  !y  with  theory.  However  the  task 
of  actually  finding  saddle-points  has  also  been  studied.  Work  In  the  early 
1950's  was  done  by  Brown-von  Neumann  [7],  Robinson  [37],  and  llanskln  [9], 
Charnes  [8]  showed  that  a  mlnlmax  problem  corresponding  to  a  constrained 
matrix  game  is  equivalent  to  a  dual  pair  of  linear  programs,  so  that  such 
techniques  as  the  simplex  method  could  be  applied.  Conversely,  in  order 
to  utilize  the  Kuhn-Tucker  theorem  [28]  and  its  generalizations  for  solving 
concave  programs,  Arrow-Hurwlcz  [2,  p.  118]  developed  a  "steepest  descent" 
method  for  locating  the  saddle-points  of  the  Lagrangian.  More  recently, 
algorithms  have  been  given  by  Dem'Janov  [14,  16],  Auslender  [3],  and  Dans- 
kln  [11  *3.  See  also  Tremolleres'  survey  paper  [56].  Algorithms  dealing  with 
max-mi n  problems  have  been  given  by  Pshenichnyi  [36],  Dem'janov  [15],  and 
Oanskln  [11]. 

The  problem  of  minimizing  a  convex  function  subject  to  constraints  has 
been  analyzed  by  various  authors  by  means  of  the  duality  theory  arising 
from  Fenchel's  conjugacy  correspondence.  This  approach,  as  expounded  In  [44], 
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rests  ultimately  on  the  duality  between  two  operations  which  combine-  a  con¬ 
vex  function  with  a  linear  transformation.  The  aim  of  this  thesis  Is  to 
analyze  constrained  mlnlmax  problems  In  a  similar  fashion  by  means  of  the 
duality  theory  arising  from  the  conjugacy  correspondence  among  saddle-func¬ 
tions.  To  accomplish  this  we  develop  for  saddle-functions  analogues  of 
these  fundamental  operations  on  convex  functions.  But  before  actually  des¬ 
cribing  our  results,  we  shall  sketch  the  two  operations  and  the  applications 
of  them  which  this  thesis  extends. 

The  simpler  of  the  two  operations  Is  to  form  the  composition  fA  of  a 
convex  function  f  with  a  linear  transformation  A.  The  other  operation  may 
be  called  "taking  the  Image  of  f  under  A."  The  resulting  function  Af  Is 
defined  by  < Af ) (x)  ■  1nf{f(y))Ay  *  x).  The  fundamental  result  connecting 
these  operations  Is  that,  under  certain  mild  hypotheses, 

(Af)*  ■  f*A*,  (1) 

where  *  of  a  linear  transformation  denotes  the  adjoint  linear  transforma¬ 
tion  and  *  of  a  convex  function  denotes  the  conjugate  convex  function. 

One  of  the  main  consequences  of  the  duality  formula  (1)  Is  the  duality 
between  the  operations  of  addition  and  inflmal  convolution  for  convex  func¬ 
tions.  This  can  be  obtained  by  taking  f  to  be  the  separable  function 

f(xi,....xm)  *  fjUj)  +...+  yy. 

where  each  fj  Is  convex  on  Rn,  and  defining  A  to  be  the  linear  trans¬ 
formation  which  sends  each  element  x  of  r"  Into  the  m- tuple  (x,...,x). 

In  this  event  fA  Is  fj  ♦...♦  fB  and  A*f*  Is  the  function 
x*  1nf{ff(xf)  fj(xj)|x*  ■  xf  ♦...+  x*), 
l.e.  the  Inflmal  convolution  of  ff,...,f*.  Formula  (1)  then  Implies  that 
under  mild  hypotheses  "the  conjugate  of  the  sum  Is  the  Inflmal  convolute  of 
the  conjugates."  This  gives  a  framework  encompassing  problems  of  the  form 
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"minimize  h(x)  subject  to  x  c  C,  where  h  and  C  are  convex."  Simply 
take  m  »  2,  let  fj  ■  h,  and  let  fg(x)  equal  0  when  x  e  C  and  +• 
otherwise. 

The  duality  represented  by  formula  (1)  Is  also  fundamental  In  the  per- 
turbatlonal  duality  theory  developed  by  Rockafellar  for  generalized  convex 
programs  [44].  Among  other  things,  this  theory  generalizes  the  classical  re¬ 
sults  about  dual  linear  programs  and  generalizes  Fenchel's  Duality  Theorem 
£21,  p.  108]  (see  also  [41,  42]  and  Stoer-WItzgall  [55]).  It  also  sheds 
light  on  the  Lagrange  multiplier  principle  for  convex  programming  and  thereby 
on  the  celebrated  Dantzlg-Wolfe  decomposition  principle  for  linear  and  convex 
programs  [44,  pp.  285-290]  (see  also  Falk  [17]  and  Lasdon  129]). 

In  the  next  three  paragraphs  we  Indicate  an  essential  difference  between 
minlmax  theory  and  convex  function  theory,  and  briefly  review  the  notions  of 
"closed"  and  "conjugate"  for  saddle- functions. 

The  principal  difference  between  convex  function  theory  and  minimax 
theory  Is  not  the  difference  between  one  and  two  arguments.  Rather  It  Is 
that  In  convex  function  theory  the  natural  object  of  study  Is  the  Individual 
convex  function,  whereas  In  minlmax  theory  the  natural  object  of  study  Is  a 
whole  equivalence  class  of  saddle-functions.  This  stems  from  the  fact  that 
there  Is  an  equivalence  relation  among  saddle-functions  with  the  property 
that  equivalent  saddle-functions  have  the  same  (lower  and  upper)  saddle-values 
and  also  the  same  saddle-points.  The  relation.  Introduced  In  [39],  Is  the 
following:  two  concave-convex  functions  K  and  L  are  said  to  be  equiva¬ 
lent  If  and  only  if  the  closures  of  the  convex  functions  K(x,*)  and  L(x,') 
coincide  for  each  x  and  the  closures  of  the  concave  functions  K(-,y)  and 
L(.,y)  coincide  for  each  y. 

Recall  that  In  convex  function  theory,  in  order  to  have  the  crucial 


formula 


(f*>*  ■  f  (2) 

hole,  one  considers  convex  functions  which  are  lower-semi -continuous ,  l.e. 
closed.  Similarly,  In  saddle-function  theory  one  considers  " regularized" 
saddle* functions  In  order  for  an  analogue  of  formula  (2)  to  hold.  A  saddle- 
function  K  defined  to  be  closed  If  and  only  If  It  Is  equivalent  to  both  Its 
concave  closure  and  Its  convex  closure ,  where  by  concave  (resp.  convex)  clo¬ 
sure  we  mev>n  the  saddle-function  obtained  from  K  by  closing  It  (In  the 
sense  of  convex  function  theory)  In  Its  concave  (resp.  convex)  argument.  It 
Is  easily  seen  that  a  saddle-function  Is  closed  If  and  only  If  every  member 
of  Its  equivalence  class  Is  closed.  It  Is  shown  In  [39]  that  the  property 
of  being  a  closed  saddle-function  Is  constructive.  In  [39]  It  Is  also  shown 
that  equivalent  closed  saddle-functions  must  be  very  nearly  equal.  Roughly 
speaking,  they  can  differ  essentially  only  at  the  "comer  points"  of  their 
"domain  of  finiteness."  In  [39]  It  Is  shown  that  each  equivalence  class  [KJ 
of  closed  saddle- functions  Is  an  "Interval"  In  the  sense  that  there  exist 
unique  members  K  and  ?  of  [K]  such  that  [K]  contains  all.  and  only 
those,  saddle-functions  IT  satisfying  KiK<lT. 

We  now  review  the  conjugacy  correspondence  for  saddle-functions,  first 
developed  In  C39).  If  K  Is  a  concave-convex  function  from  if  *Rn  to 
[-•,+•],  the  lower  conjugate  K*  and  upper  conjugate  K*  of  K  are  defined 


by 


and 


K*(x*,y*)  »  sup  Inf  (<x,x*>  +  <y,y*>  -  K(x,y)} 
y  x 


K*(x*,y*)  *  Inf  sup  (<x,x*>  ♦  <y,y*>  -  K(x,y)}. 
x  y 

These  functions  are  concave-convex.  If  K  Is  closed,  then  K*  and  K*  are 
equivalent  and  closed,  and  moreover  they  depend  only  on  the  equivalence  class 
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CK]  containing  K.  Thus,  associated  with  CK]  Is  a  well-defined  equivalence 
class  CK*]  of  closed  concave-convex  functions,  namely  the  class  containing 
K*  and  IT*.  The  class  CK*]  Is  said  to  be  the  conjugate  of  CK].  This 
conjugacy  correspondence  has  the  property  that  the  conjugate  of  CK*3  Is 
[K].  This  1$  the  analogue  of  formula  (2)  for  saddle- functions. 

With  this  review  of  general  facts  In  mind,  we  now  describe  the  results 
obtained  In  this  thesis.  We  begin  with  the  analogues  of  the  two  fundamental 
operations  described  above.  Let  K  be  a  closed  concave-convex  function  and 
let  A  be  the  linear  transformation  *  Ag  obtained  from  two  other  linear 
transformations  A-j  and  Ag  by  Aj  x  A^  (x,y)  «  (A^x.Agy).  One  of  our 
operations  consists  of  forming  an  equivalence  class  [KA]  containing  all  the 
saddle-functions  of  the  form 

(x,y)  -*•  l<!A(x,y)  »  ^(AjX.A^) 

for  %  any  member  of  CK].  A  mild  hypothesis  Is  given  which  ensures  that 
In  fact  such  a  single  class  exists  and,  moreover,  that  all  Its  members  are 
closed.  The  other  operation  Is  to  form  a  single  equivalence  class  [AK] 
containing  all  the  saddle-functions  both  of  the  form 

(u,v)  +  sup  inf  ff(x,y) 

{xjAjX  *  u>  (y | Agy  *  v) 

and  of  the  form 

(u,v)  -*■  inf  sup  £(x,y) 

{yjA^y  ■  v}  (x|AjX  ■  u) 

for  1?  any  member  of  CK].  A  mild  hypothesis  Is  given  which  ensures  that 
Indeed  such  a  class  exists  and  that  all  Its  members  are  closed.  What  Is 
surprising  Is  that  this  hypothesis  Is  precisely  the  same  as  Is  needed  to 
ensure  the  existence  of  the  class  [K*A*]  formed  by  the  first  operation 
from  CK*]  and  A*  *  Af  *  A£.  Furthermore,  it  Is  shown  that  under  this 
hypothesis  [AK]  and  [K*A*]  are  conjugate  classes.  This  is  the  analogue 
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of  forwula  (1)  for  saddle- function*. 

The  development  of  these  operations  and  fch<-*  proof  of  the  duality  between 
them  make  up  the  main  contribution  of  this  thesis*  Three  forms  of  this  du¬ 
ality  are  given.  The  most  general  version  Is  proved  in  sL  In  52  a  mere 
explicit  version  Is  given;  this  is  the  foras  v?e  find  most  useful  for  the  sub¬ 
sequent  applications.  The  formulation  In  «3  contains  the  sharpest  ccnclstfwn' 
and  requires  the  strongest  hypotheses. 

In  34  the  first  application  of  this  duality  Is  uade  In  defining  addi¬ 
tion  and  snininsftx  convolution  for  saddle- functions  and  showing  that  these  ar- 


dual  operations.  The  result 

*(Kj  +  K2)(x*y)  *  aKjKy)  *  s>K2(.i,y) 

Is  also  obtained  for  the  subd1ffersnt1.il  of  the  sum  of  two  ssddea-funciion?; . 
This  parallels  the  result  for  convex  functions  obtained  by  Rcekafellar  [38], 
Moreau  [323,  and  others.  The  duality  between  addition  and  mlniuJcx  convolu¬ 


tion  gives  a  general  framework  within  which  to  consider  problems  cf  the 
form,  "find  the  saddle-points  of,  H  with  respect  to  C  *  D,  where  H  Is  a 
saddle-function  and  C  and  D  are  convex  sets." 

From  the  first  application  we  obtain  the  following  result,  fur 
S  a  l,...,p  let  Kj  be  a  closed  concave-convex  function  on  tl  x  r,!  which 
is  not  identically  +■»  or  -•»  and  let  be  the  maxima!  monotone  ops  rater 
on  f?i+n  arising  from  the  subdlfferentlel  of  K}  (see  £44*  46]).  if  each 
R(T4)  Is  bounded,  where  R(’)  denotes  the  range  cf  an  operator,  then 

1  I 

is  maximal  monotone  and- 


£cl  R(Tj}  -  cl  R^T^. 

It  Is  known  that  this  formula  holds  whenever  the  Y.j's  ara  suhdlfferenti 
of  closed  proper  convex  functions  and  each  R(T{)  is  bounded.  Or  ihc-  <:<•' 
hand,  formula  (3)  fails  in  general  for  maximal  monotone  operators .  H-m-’ 


*v  <t 
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is  not.  known  whether  formula  (3)  holds  for  *rhi  Vary  maximal  monotoe  •; 
tors  under  the  assumption  that  the  tots  5K?^!I  are  bounded.  But.  the  v.'.r. 
th?t  it  holds  for  tfceue  maximal  monotone  operators  arising  from  saddle- 
functions  leads  one  to  conjecture  It  holds  in  general.  fhis  1>  because  s,ich 
operators ,  unlike  the  subdtfferentlals  of  convex  functions-  exhibit  most  >v 
the  pathology  of  arbitrary  maximal  monotone  operators.  Indeed,  ths  last 
fact  Is  one  of  the  main  motivations  for  studying  saddle- functions. 

In  s6  we  ma<e  a  second  principal  sj'piicatlo  of  our  fundamental  dual 
operations  in  developing  *  perturbatlonal  duality  thaory  for  generalized 
saddle  programs.  We  define  c  generalize-  v  jlk-  preg-ar  to  be  in  "objective' 
saddle- function  K  (thought  of  as  some  yJven  minima''  problem)  togetue*  uiV- 
e  particular  class  of  perturbations.  The  envi~e  program  is  given  by  ano.t: . 
saddle- function  K.  To  this  generalized  saddle  program  X  we  asrocii*.*  .*• 
dual  generalized  saddle  program  (.,  Under  mild  hypotheses  on  the  perturba¬ 
tions  in  X,  the  dual  program  L  has  a  unique  (up  it  s>|i1  valence)  "object: !vo,: 
saddle-function  The  mini max  problem  corresponding  to  I  Is  a  dua1  ta 
the  original  mini max  problem.  Optimal  solutions,  stable  optimal  solution; 
and  Kuhn--Tucker  vectors  fnr  these  dual  program*  *re  studied,  In  55,  )s  * 
subsidiary  application  of  the  fundamental  dual  operations,  ve  define  i 
p*rtlel  conjugacy  correspondence  among  closed  sad  le- font tiur.s  which  is  one- 
vC-one  md  symmetric.  5>  means  c?  this  correspondence  we  are  able  to  ascoc- 
iat.  with  t  generalized  saddle  program  and  it,  dual  a  well-deflneo  Lagranglan 
saddle- function.  We  then  give  a  characterization  of  the  primal  ami  dual 
stable  optimal  solutions  and  Kuhn-Tucker  vectors  in  t« ms  of  the  saddle-points 
of  the  Lagrangian. 

In  s7  this  perturbation*!  duality  theory  is  used  to  study  tie  problem 
of  finding  a  saddle-point  subject  to  finitely  many  convex  end  concav* 
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constraints.  Ordinary  saddle  program  are  defined  ac  a  (y-aw w-yrl.  it:  t 
such  problem,  A  question  of  particular  concern  Is  whether  or  n.-vf  3 
multiplier  principle  Holds  for  these  saddle  program.  The  analogous  question 
for  ordinary  convex  programs  (S.e.  minimising  a  convex  function  subject  U< 
finitely  many  convex  constraints)  has  a  very  satisfying  affirmative  anvoar 
(see,  for  example,  [44,  Theorem  28.13).  It  is  shown  that  one  cannot  hops 
for  a  correspondingly  general  Lagrange  multiplier  principle  for  ordinary 
saddle  programs.  This  Is  essentially  due  to  the  fact  that,  unlike  !:ho  con¬ 
vex  program  case,  the  set  of  saddle-points  of  the  Lagrangian  does  not  Tp?it 
up  Into  the  product  of  the  primal  stable  optimal  solutions  and  the  primal 
Kuhn-Tucker  vectors  (Lagrange  multipliers).  Put  another  way*  ‘the  sufsie  op¬ 
timal  solutions  and  Kuhn-Tucker  vectors  arc-  shown  to  be  in  a  certain  ?(n:se 
dependent  on  each  other. 

Finally,  in  s8  the  perturbatlonal  duality  theory  Is  specialized  in 
another  direction  to  yield  a  mlniroax  version  of  FencheVc  Duality  Tbeurwi. 

We  deal  with  dual  pairs  of  minimax  problems  of  the  foil  wing  form  (where  wt 
suppress  for  simplicity  now  the  Issue  of  the  domains  of  the-  variables)' 

(I)  Find  the  saddle-points  of  K(x,y)  -  LA(x,y), 

(II)  Find  the  saddle-points  of  L*(z,w)  -  K*A*(i,w). 

Here  k  Is  closed  and  concave-convex  on  fF  *.  Rn,  l.  is  closed  and  ccn-ax- 
concave  on  $  x  and  A  Is  a  (product)  linear  transformation  from 
if  x  Sf  to  9?  x  R^.  The  results  obtained  generalize  certain  results  of 
Rockefeller  [433*  Lebedev-Tynjanskii  [30],  and  Tynjanskli  [5/,  £81. 

It  is  known  that  many  results  in  the  theory  of  convex  f r.’ions  have 
refinements  when  polyhedralness  is  present.  For  closed  saddle -function? 
there  is  a  property  of  polyhearalnsss  which  is  preserved  under  conjugscy 
well  as  the  operations  in  s§2»  4  and  5.  Nearly  all  the  results  (n  the  thesis 


10:  Preliminaries 


In  this  thesis  we  use  mainly  the  definitions  and  notation  set  forth  » •» 
Rockafellar  [44],  and  any  terras  r.et  defined  In  the  thesis  are  Jo  dr  under*- 
stood  as  in  [44].  For  convenience  we  review  scene  definitions  hero  and  a'jso 
introduce  soce  of  our  own,  In  addition,  we  present  a  few  background  rruUs 
which  will  be  of  use  later  on. 

The  topology  taken  on  R  Is  T*)e  usual  one,  and  the  interior  and 
closure  of  a  subset  S  of  Rn  are  denoted  by  Int  S  and  cl  S,  y «•■«■:. - 
tlvely.  A  set  Is  called  affine  iff  It  Is  either  the  empty  set,  »-y 

d,  or  a  translate  of  a  linear  subspace.  The  affine  hull  or  a  subset  is  it- 
smallest  affine  set  containing  It.  If  C  Is  a  convex  subset  cf  ?!  i'1'. 
relative  Interior,  written  rl  C,  is  the  interior  of  C  with  reject  U  :'r, 
affine  hull  equipped  with  the  relative  topology. 

If  A  Is  a  linear  transformation  from  R?  to  Km,  then  /-,*  oenotj-s 
the  adjoint  linear  transformation  mapping  sf9  to  R-\. 

The  effective  domain  of  a  convex  function  f  on  R1’  Is  the  set 

dew  f  ■  (xjf(x)  <  +«»}  t 

nnd  the  conjugate  of  f  Is  the  convex  function  f*  on  Rn  given  by 

f*(x*)  »  sup{<x,x*>  -  f(x)> 

X 

(where  <-,*>  denotes  the  ordinary  Inner  product) .  Similarly,  the  effec¬ 
tive  domain  of  a  concave  function  g  on  fin  is  the  set 

dora  g  -  (x|gU)  >  —), 

and  the  conjugate  of  g  is  the  concave  function  g*  on  r”  .pvti:  by 

g*(x*)  »  1nf{<x,x*>  - 
x 

Our  multiple  use  of  the  superscript  *  should  cause  no  difficulty,  since  It 
is  always  clear  from  the  context  what  operation  is  intended. 

For  any  subset  C  of  the  function  5(-jC)  on  F'\  called  U»e 

Indicator  function  of  C,  Is  defined  by  setting  «(«!<:)  d  if 
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«  c  C  and  +*  otherwise.  Clearly  C  is  convex  iff  tf'-jC)  Is  convex, 
And  In  this  case  the  noaiuyate  of  «( * | C)  Is  denoted  by  6*(-{C)  and  is 
given  by 

«*{x*|C)  «  sup{vx»x*> j?  c  C). 

We  call  <5*( - 1 C)  the  support  f  unc - ion  of  C. 

A  concave-convex  function  on  Rr  '$  9  function  K  frou  R*r  *  ti' 

to  r.~,+-l  such  that  K(x,y)  is  a  concave  function  of  x  r.  Jr  for  each 
fixed  y  c  Rf  and  a  convex  function  of  y  «  fcr  for  w>ch  fix^r  x  c  fc.  A 
convex-concave  function  is  defined  the  sarre  cxcout  *0;  interdungiiH,  "  .r- 
cave"  with  "convex."  A  saddle-function  Is’ either  &  concave-co: vex  or  * 
ron  vex- concave  function. 

For  the  remainder  of  *0  Int  K  denote  a  concave-convex  function  on 

F  *  Rn. 

Wc  say  that  X  has  a  saadle- value,  or  that,  the  saddle -value  exists  , 
iff  the  two  q  entitles 

suji  Inf  X(x,y) 
x  y 
and 

inf  >up  K(,<,y) 

y  * 

are  equal,  in  which  case  this  rocirw  value  is  the  saddle-value  of  K.  A 
pair  i x ,y)  c  if1  *  Kp  '<  »  s-'ddle-pc ’nt  of  K  i,f 

K(x.y)  «,  F(x,y)  <  K(7.y? 

for  each  (x,y)  t  flP  *  Fn. 

Define  sunsets  dom^K  of  tf*  and  dcrs^K  of  Rr  'ey 
dow^X  "  {xjK(x,*>  is  never  -«r), 
dcm>X  ■  {y|K(*vy)  is  never  +«). 


The  product  set 


dontjK  *  do^  X  '  do m  K 
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is  the  domain  of  X.  We  say  that  X  Is  proper  iff  its  domain  is  nonempty. 
The  kernel  of  X  is  the  restriction  of  X  to  the  relative  interior  of  its 
domain.  We  say  that  X  is  simple  iff  dom  X(x,0  e  cl(dom2X)  for  every 
x  t  ritdom^X)  and  domX(*,y)  C  cl(domjX)  for  every  y  c  rlUomgX).  The 
function  cljX  obtained  by  closing  X(x,y)  as  a  concave  function  of  x 
for  each  fixed  y  Is  called  the  concave  closure  of  X.  Similarly,  the 
function  cl2X  obtained  by  closing  K(x,y)  as  a  convex  function  of  y 
for  each  fixed  x  Is  called  the  convex  closure  of  X.  If  L  Is  also  a 
concave-convex  function  on  fif*  *  Rn,  Me  say  that  X  and  L  are  equivalent 
and  write  X  ~  L  Iff  cljX  ■  cl^L  and  cl^X  •  cl2L.  The  collection  of 
all  concave-convex  functions  on  R*  *  Rn  which  are  equivalent  to  X  Is 
called  the  equivalence  class  contalnino  X  and  is  denoted  by  [X].  We  say 
that  X  is  closed  iff  cl,X~X  and  X~cl2X. 

It  Is  an  easy  exercise  to  show  that  X~  L  Implies  both  that  dom  X  » 
dom  L  and  that  X  Is  closed  Iff  L  is  closed.  Thus,  an  equivalence  class 
is  called  closed  (resp.  proper)  Iff  any  of  Its  members  Is  closed  (resp.  pro¬ 
per). 

The  function  f  on  sF*Rn  given  by  f(x,y*)  •  sup(<y,y*>  -  X(x,y)) 

y 

Is  convex  In  (x,y*)  jointly,  since  it  Is  a  polntwlse  supremum  of  convex 

functions.  Similarly,  the  function  g  on  iF  *  R°  given  by 

g(x*,y)  ■  1nf{<x,x*>  -  K(x,y))  Is  concave  In  (x*,y)  Jointly.  We  call  f 
x 

(resp.  g)  the  convex  (resp.  concave!  parent  of  X.  Notice  that  this  usage 
of  the  term  "parent1*  differs  by  some  minus  signs  from  the  original  usage 
In  Rockefeller  [43].  With  these  definitions,  (34.2)  Implies  the  following. 

THEOREM  0.1.  (jjl  f  (resp.  a)  be  the  convex  (resp.  concave)  parent  of 
X.  Then  X  is  closed  Iff  f(x,y*)  ■  -g*(x,-y*)  and  g(x*,y)  *  -f*(-x*,y), 
in  which  case  (a)  jnd  (b)  below  hold. 


u 


(a)  For,  each  t  c  CK1,  iha  /.wwax  {reAg..  ccncwg)  £i  w,t  of  K  y. 
f  (reso.  g).  Moreover,  f  snd  g  .closed,  .any 

dtirijK  *  txjf(x»y^>  <  *»  for  sera  y*}, 
dom,/.  *  iy j  <?(**!,}>)  >  •*«*  for  sore  x*!, 

U 

g(x*»y)  «  -f *(-x*9y). 

(b)  Th.;  jgigyi valance  cjfts*;  fKJ  <3KSii^  f£  »!X  sU<<.  gjplfc  ?j}{HS  A'.SS 
jaagfcfiBaYflA  litrea&ffii  k  ^  f  *  f  j&tMv.  i  <  &  1 *.» 


K(;;,y)  *  r»up<fy*,y>  ~ 


K{x»y)  *  ?nff  «-x'-<;vx.3  •  q{x*,y'} 
xfl 


itersimz.  cl  2% «  k  mi  ctfi »  k  for  £&&  1 1  l«l  *m 

K(x,y)  *  £{  x ty) 

whenever  x  t  ri(dot^K)  or  v  t  r1(dom?K). 

The  lower  conjugate  of  K„  denoted  by  K%  A»  a  function  «m 
defined  by 


5v*  £*.y*)  «  sap  Inf  <*•-.«,  x*>  -i  *yr>  ■ 

V  * 

Similarly*  the  upper  conjugate  of  K,  denoted  by  K-%  is  «  few:  her  on 


x  Sn  defined  by 


K*(x*,y*)  *  luf  sup  (<*,>;*>  t  --y ..y*>  -  ;<fx,y)}. 


y 


Prow  (37,1)  we  have  the  following  result. 

THEOREM  0,2.  Assume  Y.  is  closed,  Tfcsn  K*  end  <'•  er*:.  o*u1y».b:nt 
closed  concave -convex  functions  which  depend  only  os.  fK3.  Mor sorer,  it  I 
is  any  element  of  the  equivalence  c1e;.s  containing  K*  and  ¥*..  then 

c  *2L  “  -  4  cl  (l.  = 

i*  *  k*  r-  -  r. 

and  the  convex  (resg..  concave)  Pat  ent  of  L  I*.  the  ncc?Uvc_  of  th  cc^crti®. 
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(resp.  convex)  parent  of  K. 

The  equivalence  dess  containing  K*  and  IT*  Is  called  the  conjugate 
of  CK]  and  It  denoted  by  CK*].  Each  member  of  [K*]  Is  called  a 
conjugate  of  every  mentor  of  CK].  It  Is  Immediate  from  Theorems  0.2  and 
0.1(b)  that  (at  least  when  K  Is  closed)  K*  and  IT*  are  the  least  and 
greatest  elements  of  CK*],  respectively.  The  notation  thus  conforms  to 
that  Introduced  In  Theorem  0.1(b),  where  a  lower  (resp.  upper)  bar  indicates 
the  least  (resp.  greatest)  element  of  the  equivalence  class. 

By  (34.2.3)  the  only  equivalence  classes  which  are  closed  but  not  pro¬ 
per  are  the  one  containing  the  constant  function  +•  and  the  one  containing 
the  constant  function  — .  Since  each  of  these  two  equivalence  classes  Is 
the  conjugate  of  the  other.  It  follows  that  [K*3  Is  closed  and  proper 
whenever  CK]  Is. 

Suppose  for  now  that  K  Is  closed  and  proper,  and  let  K*  be  a  con¬ 
jugate  of  K.  By  (37.1.3)  the  saddle-value  of  K  exists  whenever 
0  c  r1(doffljK*)  or  0  c  rlCdon^K*),  and  by  (37.5.3)  a  saddle-point  of  K 
exists  when  both  these  conditions  are  satisfied.  To  use  these  facts  It 
would  be  helpful  to  have  characterisations  of  0  c  r1(douijK*)  for  J  *  1  and 
2.  These  are  furnished  by  the  next  two  lemmas. 

LEWA  0.3.  Assume  K  Js,  closed,  and  let  f  (resp.  g)  be  Its  convex 
(resp.  concave)  parent.  Then 

rlfdom^K*)  ■  U(r1(dom  g(*,y))|y  c  rl  D) 
and 

r1(dom2K*)  ■  U(r1(dom  f(x,*))|x  e  rl  C), 
where  C  ■  dom^K  and  0  »  dom2K.  These  formulas  also  hold  when  "rl"  Is 
deleted  throughout. 

PROOF.  From  Theorems  0.2  and  0.1  (a)  It  follows  that 


wgryj  » yr *yyy^ w  -t'»t»i.  u  iwn»»j  , .  >m  ytwTf^yi*  ■'■fi' »i 


W 


dom jit*  ~  A  dor,)  g 


and 


dom  g  *  U{dCflt  g (-,y)  *  {y}|y  e  Oh 
where  A  is  the  projection  (x*,y)  •+■  x*.  Henco  (-5.6)  implies 

rl  ( dcwr^  K*)  *  A  r2{dofc:  g) 

and  (6.8)  ifnplios 

r1(dom  g)  «  Ui>1(dR?n  g(*fy))  *  {y}jy  e  ri  t‘>. 

The  formulas  for  dom^K*  and  its  relative  Interior  fol  low  frw.  those .  one! 
the  other  two  formulas  are  proved  similarly 

For  the  next  lemma  mere  definition'  rrr-  needed.  ff  •'  h  a  j.'r<.<7«»’ 
convex  function  on  ftn,  the  recession,  function  of  <\  written  f  1-; 
function  on  Rn  defined  by 

{rec  f)(y)  ■  sup{f(x  >  y)  •  t'(v)  ’r  ?.  dcr-  f}» 
and  the  recession  ccng  of  f  is  the  set 

rec  cons  f  *  (yjirec  f)(y)  <  Ci}. 

The  recession  function  and  recession  cone  of  a  proper  concav?  function  a  - 
defined  similarly  by  replacing  "sup"  by  "Inf"  end  by  "yh  This  w'-xi  !■.•; 
for  these  objects  differ  from  that  In  [44]. 

Now  write  C  dnui^tC  and  0  *  douigK.  The  convex  recession  fuM\ct?un 
of  K  is  the  function  reCgK  on  Rn  c;e  fined  by 

(rt»e?K)(w)  »  sup  {(rec  K{x,-))(w)|x  e  rl  f.>. 

•ho  convex  -jccsslou  cone  of  !<  is  the  sec 

rec  •  jne..,K  «  iwj  v.rec.,K)(w)  ^  0>. 

Similarly,  the  concave  recession  function  or  v~  the  funu  ior.  rec-K  on 

■pjit 

defined  ty 

(reCjK)(3)  *  1.v»;{(rec  Kj • .y))J/)jy  ••  >'<  r), 
and  the  concave  recession  cone  of  K  is  ' 


iuiia.iUu 
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rec  conejK  ■  (z|(recjK)(z)  >,  0). 

Trivially, 

rac  conegK  ■  fi(rec  cone  K(x,*)|x  c  rl  C) 
and 

rec  conejK  »  Dtrec  cone  K(*,y)|y  e  rl  D). 

When  K  Is  closed  and  proper,  Theorem  0.1  (b)  Implies  that  the  recession 
functions  and  cones  of  K  depend  only  on  [K],  and  In  fact,  (37.2)  states 
that 

reegK  ■  «♦(  •  |  do^K*) 
and 

reCjK  •  -«*(-• jdom^K*). 

Furthermore,  when  K  1$  closed  and  proper  It  follows  from  (34.3)  and  (8.5) 
that  the  recession  cones  of  K  are  closed  convex  cones  containing  the  ori¬ 
gin.  Hence  they  are  subspaces  Iff  they  are  closed  under  multiplication  by 
-1. 

LEMMA  0.4.  Assume  K  Is  closed  and  proper.  Lftl  j  equal  1  or  2 
and  put  Sj  ■  rec  cone^K.  Then 

0  c  r1(domjK*)  Iff  SjC-Sj. 
and 

0  e  1nt(domjK*)  Iff  S j  C  (0). 

PROOF.  We  use  the  following  SUBLEMMA.  If  w*  c  Rn  and  h  Is  a  posi¬ 
tively  homogeneous  proper  convex  function  on  Rn,  then  the  following  two 
conditions  are  equivalent: 

(a)  Vw  c  Rn.  <w,w*>  <.  h(w)  with  strict  Inequality  for  each  w 
such  that  -h(-w)  p  h(w) ; 

(b)  Vw  c  Rn,  h(w)  <,  <w,w*>  Implies  h(-w)  <  <-w,w*>. 

PROOF  OF  SUBLEMMA.  Assume  (a)  and  suppose  h(w)  <_  <w,w*>.  Then  actually 
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h(w)  ■  <w,w*>.  If  we  had  -h(-w)  f  h(w),  then  (a)  would  Imply  <w,w*>  <  h(w), 
a  contradlctlor .  Thus  -h(-w)  •  h(w)  ■  <w,w*>  and  (b)  Is  proved.  Converse¬ 
ly,  assume  (b)  Jand  let  w  be  given.  If  h(w)  <  <w,w*>,  then  (4.7.2)  and 
(b)  Imply  -h{»  ^  <,  h(-w)  <  <-w,w*>  and  hence  <w,w*>  <_  h(w).  Suppose 
-h(-w)  f  h(w).  By  (4.7.2)  we  have  -h(-w)  <  h(w)^  If  h(w)  <  <w,w*>,  this 
would  Imply  -h|-w)  <  <w,w*>  while  at  the  same  time  from  (b)  we  would  have 
<w,w*>  <  -h(-wl.  Therefore  <w,w*>  <  h(w)  whenever  -h(-w)  t  h(w),  and  (a) 

Is  proved.  i 

Define  h  »«*(•  (dorngK*).  By  (13.1)  and  the  Sublemma,  0  e  ri(dom2K*) 
iff  for  each  w  eV1,  h(w)  <:  0  Implies  h(-w)  <_  0.  By  (13.1)  we  also  hava 

that  0  c  IntfdomgK*)  Iff  for  each  w  c  Rn,  h(w)  <  0  Implies  w  3  0.  Tne 

assertions  for  j  *  2  follow  from  these  equivalences  and  the  fact  that 

h  *  recgK.  The  assertions  for  J  *  1  follow  similarly,  using  the  fact  that 
recjK  •  -«*(  —  |do*jK*). 

In  view  of  Lemmas  0.3  and  0.4  and  the  observations  preceding  Lemma  0.3 
concerning  the  existence  of  saddle-values  and  saddle-points,  formulas  are 
given  In  if 2  and  4  for  the  parents  and  recession  functions  of  the  saddle- 
functions  resulting  from  the  operations  developed  there.  This  1$  done  In 
Theorem  2.4,  Corollary  4.6.2,  and  Lemmas  2.5  and  4.7.  By  combining  these  re¬ 
sults  with  Lemmas  0.3  and  0.4  the  reader  can  easily  state  existence  theorems 
as  needed. 

The  next  lemma  Is  used  In  !S2  and  4  to  dualize  various  conditions.  Also, 
taking  Lj  *  {0}  In  this  lemma  yields  the  assertion  "0  e  rl (donate*)  iff 
c  -Sj"  of  Lemma  0.4. 

LEMMA  0.5.  Let  Lj  (resp.  LJ  be  a  subspace  of  ff  (resp.  Rn).  and 
assume  K  j_s  closed  and  proper.  Then  for  j  •  1  and  2  the  following  con- 
dltlons  are  equivalent: 


(«j)  Lj  n  rKdonijK*)  *  *; 

(bj)  Lj  A  (rec  conejK)  Is  a  subspace; 

(Cj)  Lj  A  (rec  conejK)C  -(rec  conejK). 

PROOF.  We  prove  only  that  (82)*  (b2)  and  (C2)  are  equivalent,  as  the 
proof  for  j  *  1  is  similar.  By  the  mmarks  preceding  Lenina  0.4  and  the 
fact  that  L2  Is  a  subspace,  (bg)  Is  equivalent  to  (C2).  Write  0*  *  don^K 
and  L  ■  By  (11.3),  (82)  falls  Iff  there  exists  a  hyperplane  separating 
L  and  D*.  By  (11.1)  this  occurs  Iff  there  exists  a  w  e  Rn  such  that 
1nf{<y*,w>)y*  t  L)  >  sup{<y*,w>|y*  c  D*} 
and 


sup{<y*,w>|y*  e  L)  >  1nf{<y*,w>|y*  e  0*}. 


But  since 

sup{<y*,w>|y*  e  0*}  ■  (rec2K)(w) 
and 

1nf{<y*,w>Jy*  t  1}  »  f  0  If  w  e  L 

*  -  IF  w  il  , 

this  means  that  (a2)  falls  Iff  there  exists  a  w  c  L  such  that 
(rec2K)(w)  ±  0  and  (re c^KM-w)  >  0.  Therefore  (a2)  holds  Iff  for  each 
wet1,  (rec2K)(w)  <. 0  Implies  (rec2K)(-w)  ^  0.  But  this  last  condition 
Is  the  same  as  (c2). 

Define  K  to  be  polyhedral  Iff  It  Is  closed  and  either  Its  convex  or 
Its  concave  parent  Is  polyhedral.  By  Theorem  0.1,  if  K  Is  polyhedral  and 
L  Is  equivalent  to  K,  then  L  is  polyhedral.  Thus,  an  equivalence  class 
Is  called  polyhedral  Iff  any  of  Its  members  is  polyhedral. 

From  Theorem  0.2  follows  the  Important  fact  that  [K*]  Is  polyhedral 


whenever  [K3  Is  polyhedral.  Polyhedral  ness  Is  also  preserved  by  each  of 
the  operations  developed  In  if  2,  4  and  5. 


si:  A  General  Theorem 


The  goal  of  this  section  Is  to  prove  Theorem  1.1,  which  concerns  the 
following  question.  Let  K  be  a  saddle-function  on  r”1  *  Rn,  let 
Aj:  R**  R™  and  A^  Rq  R°  be  linear  transformations,  and  suppose  that 
all  the  saddle-functions  of  the  form  (u,v)  ♦  E^u^v),  where  IT  Is  equi¬ 
valent  to  K,  belong  to  a  single  equivalence  plass.  What  can  be  said  about  * 
the  conjugate  (l.e.  "dual")  equivalence  class?  Theorem  1.1  describes  this 
class  explicitly.  In  12  this  duality  Is  developed  In  more  detail  as  two  dual 
operations  on  equivalence  classes. 

THEOREM  1.1.  Let  K  be  a  concave-convex  function  on  Rf"  *  Rn  and  let 
A  *  Aj  *  Ag,  where  Aj :  rP  •*  RP  and  A2*.  Rq  Rn  are  linear  transformations. 
Assume  K  Is  closed  and  that  there  exists  a  closed  concave-convex  function 
H  on  Rp  x  such  that  Ea  js  equivalent  to  H  whenever  E  equiva¬ 

lent  to  K.  Define  J1  and  J2  on  fP  x  Rp  |jy 

Ji(u*,v*)  -  sup  Inf  K*(x*,y*) 

(x*|A|x*  »  u*)  (y*|AJy*  »  v*} 

and 

J9(u*,v*)  *  Inf  sup  E*(x*,y*), 

4  {y*|AJy*  ■  v*)  {x*|Afx*  ■  u*) 

and  let  J  be  arjy  concave-convex  function  on  Rp  x  Rq  such  that 

Then  J  is  simple  and  satisfies 

dgC^J  ■  H*,  cl^clgJ  ■  H*, 

dom(c12cl1J)  ■  dom  H*  ■  dom(cl]Cl2J), 

where 

dom  H*  C  c1(A*dom  K*). 

J|  H  Is  proper,  then  J  Is  proper  and  has  the  same  kernel  as  H*. 

From  Theorem  0.2  and  Lemma  1.6  (below)  It  follows  that  In  the  theorem 
J  can  be  taken  to  be  either  0^  or  J2. 

The  proof  of  this  theorem  uses  seven  lemmas.  The  assertion  of  Lemma  1.2 


21 


was  first  noted  In  Rockafeilar  [39]  and  the  proof  given  is  the  one  indicated 
there.  Lemma  1.3  was  suggested  by  the  proof  of  (34.5).  Lerma  1.4  was  first 
proved  In  [391. 

LEMMA  1 .2.  Le£  K  be  a  concave-convex  function.  Then  the  lower  con- 

jugate  of  K*  1s_  cl?cl,K,  and  the  upper  conjugate  of  £*  is  cl ^ cl ?K. 

PROOF.  Observe  that  for  any  convex  function  f, 

(cl  f)(y)  ■  sup  Inf  (<y  -  w,y*>  +  f(w)l 
y*  w 

follows  trivially  from  the  fact  that  cl  f  *  (f*)*.  Similarly, 

(cl  g)(x)  «  inf  sup  {<x  -  z,x*>  +  g( z) } 

X*  z 

for  any  concave  function  g.  If  H  denotes  the  upper  conjugate  of  1C*, 

then  these  facts  together  with  the  definitions  Imply 

H(x,y)  *  inf  sup  (<x*,x>  ♦  <y*,y>  -  inf  sup  {<z,x*>  +  <w,y*>  -  K(z ,w) }} 
x*  y*  z  w 

-  inf  sup  {<x  -  z,x*>  +  sup  inf  {<y  -  w,y*>  +  K(z,w)}} 
x*  z  y*  w 

*  inf  sup  (<x  -  z,x*>  +  (clJ<}(z#y)} 

X*  z 

*  (cl1cl2K)(x,y). 

The  other  assertion  is  proved  similarly. 

LEMMA  1.3.  Let  K  be  a  concave-convex  function.  Then  dom^K  = 
dom^clgK),  dom^K  cz  dou^cl^K),  and.  domcl2K(-,y)  =  dorn^K  for  every  y. 
If_  dom^K  f  t  and_dom  K( x , - )  Cl  cHdorr^K)  for  every  x  e  ri(dom^K),  then 
actually  dom^K  C  domp{cl?K)  d  c1(dom?K)  and  moreover  cl^K  agrees  wi  th 
K  on  the  set  ri(dom  K).  Parallel  assertions  hold  concerning  cljK. 

PROOF.  By  the  definition  of  dom^K  ana  ao.u,fc,  when  x  i  dom^K  the 
convex  function  K(x,’)  somewhere  has  the  value  whereas  when  x  e  dom,K 

the  function  K(x,‘)  is  never  -•»  and  its  effective  domain  includes  dom^K. 
Thus  (clgKMx,-)  is  the  constant  function  -»  when  x  t  dom^K,  whereas 
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when  x  c  dom^K  it  is  never  -«  and  Its  effective  domain  includes  dun^K. 
This  shows  that  dom^clgK)  ■  domjK  and  dom2(cl2K)  D  dom^K,  and  in  fact 
that  domjK  Is  the  effective  domain  of  every  one  of  the  concave  functions 
(c!2K)(',y).  Now  assume  that  dom  K(x» * ) c:  cl(dom2K)  for  every 
x  c  ri(dom^K).  Since  dom?Kc  dom  K(x,-)  always  holds,  (6.3.1)  implies 

ri{dom  KCx,*))  *  r1(dom,K)  (II 

for  every  x  e  rl(dom^K).  Thus,  for  each  x  c  ri(dom^K)  the  convex  function 
K(x,*)  agrees  with  Its  closure  on  rifdotr^K).  That  is,  K  agrees  .vi th 
cl on  the  set  rl(dom^K)  *  rl ( dom^K)  *  r1(dom  K) .  Final  1>  ,  assume 
dom^K  f  tf.  By  (6.2)  we  can  pick  some  x  t  ri(dom^K).  Then  x  c  dam^clgK) 
Implies  dom(c1?K)(x,*)  CL  cl (dom  K(x,-))»  and  equation  (1)  and  (6.3.1)  imply 
cl(dom  K( x , * ) )  a  cl(dom^K).  Thus,  for  any  y  c  don^Ccl^K),  (cl2K}(x,y)  •  +-> 
implies  that  y  e  dom(cl2K)(x,* )  and  hence  y  e  clCdon^K).  This  shows 
dom2(cl2K)C  cl(dom?K). 

LEMMA  1.4.  Let  K  be  a  concave-convex  function.  Then  cl^K  arid 
cljK  are  simple. 

TROOP.  It  suffices  to  prove  clgK  Is  simple.  This  requires  shewing 
(i)  dom(cl,K)(*,y)  ccHdom^cloK))  whenever  y  c  rlfdom^cl^)),  and  (li) 
dom(cl2K)(x,-)C  cl(dom2(cl2K))  whenever  x  c  ri (dom^c^K) ) .  lemma  i.- 
Implies  dom,(cl2K)  »  dom^K  and  dom(d2K)(* ,y)  =  dom,K  for  every  y. 

This  establishes  (i).  Since  a  concave  function  agrees  with  its  closure  on 
the  relative  Interior  of  Its  effective  domain  (see  (7.2)  and  (7.4)),  Lenma 
1.3  also  implies  (cl2K)(x,y)  3  cl ( (cl^K) ( ' ,y))(x)  n  (c1]d2K)(  x,y)  whenever 
x  e  ri(donij(cl2K)).  Applying  Lemma  1.3  once  more  yields  dom(cljCl2K)(x,*)  * 
<Jom2( clgK)  for  every  x.  Combining  these  facts  establishes  (ii). 

LEMMA  1.5,  Let  K  oe  a  concave-convex  function.  If.  rl ( doni( c  1  ?  )> 

-i(dom(cl^cl?K)),  them  K  is  simple. 
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PROOF.  Assume  there  exists  an  x  t  rl (dom^K)  such  that  dom  K(x,*)  cf: 
cl(de*2K).  Then  there  exists  a  y  such  that  K(x,y)  <  +»  but 
y  t  cl(dom2)C).  Latina  1.3  Implies  dom(cl2IC) ( •  ,y)  ■  dom^K.  Since  an  Improper 
concave  function  euqals  +-  everywhere  on  the  relative  Interior  of  Its  effec¬ 
tive  domain  (by (7.2)) ,  (cl2K)(x,y)  <  K(x,y)<  +•  Implies  that  (cl2X)(*.y) 

Is  never  *».  Hence  y  c  dom^clgK).  From  Lemma  1.3  It  follows  that 
don^K  c  doeigUlgK)  ■  dom^djCljK)  <£  c1(dom2K). 

By  (6.3.1)  this  Implies 

r1(dom2(c1jCl2K))  A  H(dom2K).  (1) 

By  (33.1.1)  cljK  Is  a  concave-convex  function.  Hence  Lemma  1.3  Implies 

dom2K  ■  dom2(cljK)  c  dom2(c12cl jK) 

and  dom^K  c  dom^cl^K).  Since  x  e  rl(dom^K),  this  shows  dom^cljK)  f 
and  by  Lemma  1.4  cl^K  Is  simple.  Hence  Lemma  1.3  also  Implies 

dom2(c12c1jK)  C  cKdom^cl^)). 

These  facts  together  with  (6.3.1)  Imply  that 

r1(dom2(c12c1jK))  »  r1(dom2K).  (2) 

From  (1)  and  (2)  It  follows  that  r1(dom2(cl2cljK))  •  r1(dom2(cl  jd2K)) 

Implies  dom  K(x,<)  C  c1(dom2K)  for  every  x  c  rl(dom^K).  The  other  con¬ 
dition  for  K  to  be  simple  can  be  established  similarly. 

LEMMA  1.6.  Let  L  be  a  concave-convex  function  on  if  *  Rn  and 
B  *  B|  x  be  £  linear  transformation  from  r"1  x  Rn  to  Rp  *  Rq.  Define 

J.(z,w)  -  sup  Inf  L(x,y) 

{x|B-| x  ■  z)  (y iB^y  »  w) 

and 

J2(z,w)  ■  Inf  sup  L(x,y) 

{y|B2y  ■  w)  (x |B|X  ■  z) 

for  every  (z,w)e  Rp  x  Rq.  Then  Jj  and  J2  are  concave-convex  functions 
on  Rp  x  R^. 
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PROOF.  l«t  z  c  Rp  be  fixed  end  write  f  •  Jj(z,*)*  Then  f(w)  * 
sup{(B2L(x,‘))(w)|8jX  ■  *).  Since  (5.7)  Implies  Bgl(x,*)  Is  e  convex  func¬ 
tion  for  eich  x,  (5.5)  Implies  f  Itself  Is  convex.  Now  let  w  e  Rq  be 
fixed  end  write  g  ■  Jj(*,w).  The  function  k(x)  •  1nf{L(x»y) iB^y  •  w)  Is 
concave  by  (5.5) ,  so  g  •  Bjk  Is  concive  by  (5.7).  This  shows  Jj  Is  con¬ 
cave-convex.  Similarly,  (5.5)  and  (5.7)  Imply  Jg  Is  concave-convex. 

LEMMA  1.7.  Let  the  notation  be  as  In  Lemma  1.6.  Then  L*B*<J|  and 
Jf  <  I*B*.  where  B *  ■  Bf  *  BJ. 

PROOF.  For  J  ■  1  and  2  let  Rj  denote  the  range  of  Bj.  Observe 
that  J2(’,w)  Is  constantly  4*  whenever  w  i  Rg,  and  Jg(z,w)  ■  -»  when¬ 
ever  w  c  Rg  but  z  i  Rj.  These  facts  and  the  definition  of  lower  conjugate 
yield 

JJ(z*,w*)  ■  sup  Inf  (<z,z*>  ♦  <w,w^  -  J2(z,w)} 


*  sup  Inf  (<z,z*>  ♦  <w,w*>  -  J2(z,w)) 
we  Rg  zcRj 

■  sup  Inf  sup,.  Inf  ,  (<z,z*>  ♦  <w,w*>  -  L(x,y)} 
wcRg  zcRj  yeBg  w  xeBj  z 

>,  sup  sup  .  Inf  Inf  ,  {<z,z*>  ♦  <w,w*>  -  L(x,y)) 

"  wcRg  ycBg  w  zcRj  xeBj  z 

■  sup  sup  .  Inf  Inf  .  {<B,x,z*>  +  <8*y,w*>  -  L(x,y)) 

y  yey  ♦  Bj1 0  x  xcx  ♦  Bjl0  1  c 

■  sup  Inf  (<x,Bfz*>  ♦  <y,B*w*>  -  L(x,y)) 

y  x  1  c 

•  i*(BJz*,B|W*) 

■  L*8*(z*,w*), 

where  the  Inequality  follows  from  (36.1).  This  proves  the  first  Inequality, 
and  the  second  Is  proved  similarly. 

LEMMA  1.8.  Let  K  and  A  bgliiS  Theorem  1.1.  write  C  ■  domjK  and 
0  ■  domgK,  and  Jet  f  (resp.  g)  b$  Uu  convex  (resp.  concave)  parent  of  K. 
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If  K  i£  concave-closed.  thtn 

dom^KA)*  •  U(dem  c1(Afg(«»y))|y  c  D  A  range  Aj). 

If  K  Jif  con vex- closed.  then 

doa^t^O*  ■  U(dom  c1(A|f(x,*))|x  c  C  A  ring*  Aj). 

PROOF.  He  prove  only  the  first  formula,  at  the  second  Is  similar. 

Assume  K  Is  concave-closed.  Then  (16.3)  liplles  (K^.yjAj)*  •  c1(Ajg(>,y)) 

for  every  y.  Also*  If  yd  0  then  K(*,y)  Is  +•  somewhere,  which  Implies 

that  o(*»y)  ■  K(«,y)*  end  hence  d(A|g(*,y))  are  constantly  Free 

these  facts  It  follows  that 

(K£)*(u*,v*)  •  sup  Inf  (<u,u*>  ♦  <v,v*»  -  KfAjU./Uv)) 
v  u  4 

■  sup  (<v,v*>.  ♦  (K(*,A2v)A1)*(u*)) 

•  vfj^i0  ♦  c1(A|g(»,A2v))(u*)). 

Nonce  u*cdoi1(J^)*  Iff 

V  v*.  3  v  *  Aj*D  such  that  u*  c  dom  cHAfg^.Ajv)), 
and  this  occurs  Iff 

Bye  0  Arango  Aj  such  that  u*  «  doro  c1(Af*g(‘,y)). 

The  first  forswla  Is  Immediate  fro*  this. 

PROOF  OF  THEO&M  1.1.  Since  K  Is  closed,  Theorme  0.2  Implies  that 
the  lower  conjugate  of  IT*  Is  K  and  tha  upper  conjugate  of  K*  Is  IT. 

Thus,  Lean*  1.7  Implies  and  7f<10l.  From  and  (38.1) 

It  follows  that  <  J*  <  7*  <,  7f.  Therefore 

JJA  <  i*  *3*  < 

Together  with  Theorms  0.1  end  0.2,  this  Implies  that  the  lower  conjugates 
of  2*  and  H  coincide  and  the  upper  conjugates  of  7*  and  H  coincide, 
lut  hy  Lemma  1.2  this  means  that 

c^cljJ  •  H*  and  cljtfjJ  ■  ff#. 
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Since  equivalent  saddle-functions  have  the  sat*  domain,  this  implies  that 

domfc^cljJ)  *  dom  H*  ■  dom(cljd2J)  0) 

By  the  hypothesis  on  H  and  Theorems  0.1  and  0.2  we  have  that 

dom  H*  ■  dom^(KA)*  *  dom2(P)*.  (2) 

Let  f,  9,  C  and  0  be  as  In  Lemma  1.8.  Since  K  Is  closed.  Theorem 
0.1  Implies  that  £  Is  concave-closed,  K  Is  convex-closed,  and  both  IT 
and  K  have  the  same  parents  as  K.  Hence  Lemma  1.8  Implies,  for  example, 
that 

dom2(P)*  *  U(dom  cl(A$f(x,*))  |x  e  C  A  range  A2>. 

Since 


dom  cl(AJf(x,*))  <Z  d(dom  A|f(x,*))  ■  cl(AJdom  f ( x . * ) ) 
for  each  x,  we  conclude  that 

dom2(P)*  c  c1(AjU(dom  f(x,*)|x  e  C  A  range  Aj}) . 
But  by  Lemma  0.3  we  know  that 


dom2K*  ■  U{dom  f(x,*)|x  c  C). 

Using  (2),  It  follows  that 

domjH*  C  cl  (AJdomjK*) 

for  J  ■  2.  The  corresponding  inclusion  for  j  ■  1  follows  from  (2)  simi¬ 
larly.  Hence 


dom  H*  C  cl(A*dom  K*). 


From  (1)  and  Lemma  1.5  it  follows  that  J  is  simple.  Now  assume  H  Is 
proper.  Then  the  set  In  (1)  Is  nonempty.  If  dom^J  were  empty,  then  cl2J 
and  hence  cljd2J  would  be  constantly  — ,  contradicting  the  fact  that 
dom^(c1^cl2J)  is  nonempty.  Hence  dom^J  f  t.  A  similar  argument  shows 
dom2J  f  it,  so  that  J  is  proper.  Since  J  is  also  simple,  it  follows 
from  (6.3.1)  and  Lemmas  1.3  and  1.4  that  J  has  the  same  kernel  as  MjJ, 
which  In  turn  has  the  same  kernel  as  cl2c1j0.  Then  since  clgCljJ~M*t 
Theorem  0.1  (b)  Implies  that  J  has  the  same  kernel  as  H*. 


$2:  Two  Dual  Operations 


In  this  section  we  develop  two  fundamental  operations  Involving  linear 
transformations  and  equivalence  classes  of  closed  proper  saddle- functions. 
Specific  conditions  are  given  under  which  the  operations  can  be  performed, 
and  the  operations  arc  shown  to  be  dual.  Various  results  are  proved  con¬ 
cerning  the  equivalence  classes  resulting  from  these  operations.  The  sec¬ 
tion  concludes  with  examples  showing  that  the  conditions  under  which  the 
operations  can  be  performed  cannot  In  general  be  weakened. 

The  first  operation  we  develop  Is  analogous  to  that  of  composing  a  con¬ 
vex  function  with  a  linear  transformation.  Let  K  be  a  closed  proper  con¬ 
cave-convex  function  on  fP  *  Rn  and  let  A  ■  Aj  *  be  a  linear  transfor¬ 
mation  from  Rp  x  Rq  to  R01  *  Rn.  We  seek  a  condition  on  K  and  A  ensur¬ 
ing  the  existence  of  an  equivalence  class  of  closed  proper  saddle-functions 
which  contains  every  function  of  the  form  KA  for  K  c  [K].  Such  a  condition 
is  given  In  Theorem  2.2.  When  this  equivalence  class  exists.  It  will  usually 
be  denoted  by  [KA]. 

LEMMA  2. 1 .  Let  K  be«  concave-convex  function  on  if  x  Rn  aod  let 
A  ■  Aj  x  Aj>  be  a  linear  transformation  from  Rp  *  Rq  to  R01  *  Rn.  Then  KA 
li  a  concave-convex  function  on  Rp  *  Rq,  and  A”l(dom  K)  c  dom  KA.  The 
inclusion  can  be  strengthened  to  equality  Jf  I  1$  closed  and  proper  and 
range  A  r\  r1(dom  K)  t  i. 

PROOF.  Write  dom  K  ■  C  *  D.  By  (5.7),  KA  Is  concave-convex.  If 

u  c  A^C,  then  K(AjU,*)  Is  never  -•  and  hence  KA(u,»)  «  K(AjU,*)A2  Is 

never  -•.  This  shows  A^C  C  dom^KA.  Similarly  A^D  d  dom^KA.  Now 
assume  K  Is  closed  and  proper  and  range  A  ArHdom  K)  f  t.  If  u  i  A^C, 
then  (34.3)  Implies  K(A]U,*)A.,  equals  -»  everywhere  on  rl  D  and  hence 

K(A|U,*)A2  equals  -»  everywhere  on  A^ri  D).  Since  A^l(r1  D)  +  4  by 

hypothesis,  this  shows  dom^KA  C  AjlC.  Similarly  dom^KA  C  A^D. 
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THEOREM  2.2.  K  ge  &  closed  proper  concave-convex  function  on 
flP  *  Rn»  |n&  A  •  A]  *  Ag  fci  a  linear  transformation  from  Rp  x  R** 
flP  x  Rn  such  that  ranoe  AArKdom  K)  i  4.  Then  the  collection 
(KA|K  c  [K])  of  saddle-functions  Is  contained  In  an  equivalence  class  CH] 
flt  filfitSi  gCfflBtl  iMfclm  fiH  &  *  Rq  Having  domain 

A*l(domK).  MflCjflttt, 

H  ■  JKA.  IT  ■  KA, 
r1(dom  H)  *  A^rlfdom  K), 
cl  (dors  H)  -  A~Ic1(don  K). 

PROOF,  lemma  2.1  Implies  KA  and  KA  are  proper  concave- convex  func¬ 
tions  on  Rp  x  Rq  with  domain  A"l(dom  K).  From  Theorem  0.1  (b)  It  Is 
clear  that  a  closed  proper  saddle-function  Is  the  least  member  of  Its  equi¬ 
valence  class  Iff  It  1$  convex-closed.  Now  It  follows  routinely,  using  (6.7) , 
(34.3)  and  (9.5).  that  KA  satisfies  the  six  conditions  of  (34.3)  and  more¬ 
over  Is  convex- closed.  Hence  KA  Is  closed  and  Is  the  least  member  of  Its 
equivalence  class.  Similarly,  KA  1$  closed  and  Is  the  greatest  member  of 
Its  equivalence  class.  According  to  (34.4),  two  closed  proper  saddle-func¬ 
tions  are  equivalent  Iff  they  have  the  same  kernel.  Suppose 
(u,v)  c  r1(-A‘l(dom  K)).  By  (6.7)  this  means  A(u,v)  e  r1(dom  K).  Since  K  and 
IT  are  equivalent  closed  proper,  KA(u,v)  ■  KA(u,v).  This  shows  KA~KA  and 
hence  [KA]  ■  [KA]  ■  [H].  If  K  *  [K],  then  K  <  K  <  K  Implies 
KA<KA  <KA  and  hence  KA  c  [H]  (Theorem  0.1(b)).  The  formulas  for 
r1(dom  H)  and  cl(dom  H)  are  Immediate  from  (6.7). 

THEOREM  2.3.  Let  K  and  A  be  as  In  Theorem  2.2.  Then 

a(KA)(u,v)  »  A*aK(A(u,v)) 
for  each  (u,v)  t  Rp  x  R^t  end 

rKA’^dom  K))  C  dom  a(KA)  C  A-1(dom  K). 
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PROOF.  The  Inclusions  are  Immediate  from  (37.4)  and  Theorem  2.2.  It 
follows  that  the  Identity  holds  trivially  when  (u,v)  i  A~!(dom  K).  Suppose 
(u,v)  c  A"1  dom  K.  By  the  definitions,  (u*,v*)  t  a(KA)(u,v)  Iff 
u*  c  3(K(*,A2v)A  )(u)  and  v*  c  a(K(AjU,*)A2)(v).  Now  by  (34.3), 

AjU  c  dom|K  Implies  that  K(AjU,<)  Is  a  proper  convex  function  with 
r1(dom  K( A^u, • ) )  ■  rlfdon^K).  Hence  range  A2n  r1(dom2K)  t  A  and  (23.9) 
Imply  that  v*  c  3(K(A1u,«)A2)(v)  Iff  v*  c  ApKfAjU.'HAgV),  l.e.  Iff 
v*  c  AJa2K(A(u,v)).  Similarly,  u#  c  3(K(»,A2v)Aj)(u)  Iff 
u*  c  AfdjK(A(u,v}).  The  Identity  follows. 

THEOREM  2.4.  Let  K  and  A  be  as  In  Theorem  2.2.  let  f  (rest),  q) 
denote  the  convex  (reso.  concave)  parent  of  K,  and  let  h  (resp.  k)  de¬ 
note  the  convex  (reso.  concave)  parent  of  KA.  Then 

h(u,v*)  -  (A|f(A1u,-))(v*) 

|Qd 

k(u*,v)  ■  (Afg(.,A2v))(u*). 

PROOF.  Suppose  u  <  dom^KA.  Then  h(u,»)  Is  constantly  +■.  Also, 

A|U  t  dom-jK  Implies  ffA^u,*)  Is  constantly  +»  and  hence 
(A2f(A|U,«))(v*)  ■  1nf(f(A1u,y*)|AJy*  «  v*>  ■ 
for  every  v*.  Now  suppose  u  c  dom^KA.  By  (34.3),  A^u  c  dom^K  Implies 
KfAjU,*)  Is  a  proper  convex  function  with  r1(dom  K(AjU,*))  ■  r1(don»2K). 

Thus  from  (16.3),  Theorem  0.1  (a)  and  range  AgHrlfdon^K)  A  A  It  follows 
that  h(u,v*)  ■  (K(A1u,*)A2)#(v*)  ■  (AfCfAjU,  •)*)(**)  ■  (A}f(AjU,‘))(v*)  for 
every  v*.  This  proves  the  first  Identity.  The  second  Is  proved  similarly. 

COROLLARY  2.4.1.  Let.  K  and.  A  be  as  In  Theorem  2.2.  If  K  i£ 
polyhedral ,  then  KA  Is  polyhedral. 

PROOF.  Let  Ip  and  IR  denote  the  Identity  transformations  on  Rp 
and  A*1,  respectively,  and  let  f  and  h  be  as  In  Theorem  2.4.  Then 
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h  ■  (Ip  *  AJ)(f(Aj  k  in)).  Hence  (19.3.1)  Implies  that  h  Is  polyhedral 
If  f  Is  polyhedral.  Since  KA  Is  closed  by  Theorem  2.2,  this  concludes 
the  proof. 

COROLLARY  2.4.2.  Let  K  and  A  be  as  In  Theorem  2.2.  Then 

dom(KA)*C  A*dom  K*. 

In  particular.  If  f  (jrgsg.  g)  denotes  the  convex  (resp.  concave)  parent 
of  K,  then 

rlfdom^KA)*)  ■  AfU{r1(dom  g(*,y))|y  e  range  A2  n  r1(  dom2IC)) 

and 

r1(dORi2(KA)*)  •  AjU{r1(dom  f(x,»))|x  c  range  A]  A  rl(domjK)}, 
where  these  formul as  also  hold  when  "r1“  l£  deleted  throughout. 

PROOF.  By  Lemma  0.3. 

lemma  2.s.  Lfil  *  iM  a  fefc  ml  la  Itemn  LL  Iten 

(reejKAMu)  *  1nf{(rec  K( •  .yJJCA^u) |y  c  range  A2A  rlfdomgK)} 
and 

(rec2KA)(v)  ■  sup{(rec  K(x,»))(A2v)|x  c  range  Aj  Ar1(dom|K)}. 

PROOF.  By  definition  and  Theorem  2.2, 

(rec2KA)(v)  •  sup{(rec(KA)(u,»))(v)|u  c  Aj'rKdomjK)). 

■  sup{(rec  K(x,*)A2)(v)|x  e  range  A^A  rUdom^K)}. 

If  x  c  rl(dorajK),  then  (34.3)  and  (9.5)  Imply  (rec  Mx.OAgMv)  ■ 

(rec  K(x,»))(A2v).  This  proves  one  formula,  and  the  other  Is  proved  similarly. 

THEOREM  2.6.  Let  K  and  A  be  as.  In  Theorem  2.2.  Then  for  J  ■ 

1  and  2, 

cl(donij(KA)*)  ■  cKAJdomjK*) 
reCj(KA)  ■  (reCjK)Aj. 

PROOF.  By  (37.2),  rec2(KA)  -  «♦( •  |do«2(KA)*)  and 


Iff 
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(rec2K)A2  ■  «*( Ag- 1 donigK*) .  Now  apply  (16.3.1)  and  (13.1.1).  The  assertion 
for  J  ■  1  Is  proved  similarly. 

Next  we  develop  an  operation  analogous  to  that  of  taking  the  Image  Af 
of  a  convex  function  f  under  a  linear  transformation  A.  Suppose  K  Is 
a  concave-convex  function  on  Rf”  *  Rn  and  A  »  Aj  x  A2  Is  a  linear  trans¬ 
formation  from  flF  x  Rn  to  RP  *  R^. 

Me  seek  a  condition  on  K  and  A  ensuring  that  all  the  functions  on 
rP  *  Rq  either  of  the  form 

(u,v)  ♦  sup  Inf  ff(x,y) 

(xJAjX  ■  u}  {y|Agy  »  v> 

or  of  the  form 

(u»v)  ♦  Inf  sup  K(x,y), 

{ylAgy  ■  v}  {x  A^x  *  u> 

for  K  e  [K],  belong  to  a  single  equivalence  class  of  concave- convex  func¬ 
tions  on  Rp  x  By  analogy  with  the  operation  In  the  convex  function  case, 
this  equivalence  class  (when  It  exists)  will  usually  be  denoted  by  [AK]. 
Theorem  2.8  gives  a  condition  which  guarantees  that  [AK]  exists  and,  more¬ 
over,  that  all  of  Its  members  are  closed  and  proper,  and  that  Its  conjugate 
Is  [K*A*]. 

LEMMA  2.7.  yjt  K  be  a  closed  proper  concave-convex  function  on 
rf™  x  Rn,  and  let.  A  ■  Aj  x  Ag  jbe  a  linear  transformation  from  Rm  x  Rn  to 
rP  *  Rq.  Assume  range  A*nr1(dom  K*)  f  i.  Then 

(7*A*)*(u.v)  ■  sup{c1(A2K(x,*))(v)|A1x  •  u}, 
where  the  suoremum  can  be  taken  over  lust  those,  x  la  donate  iygh  that 
A^x  *  u,  and 

(K*A*)*(u,v)  «  1nf{cl(Aj^( - ,y))(u) |Agy  •  v), 
where  the  Inflmum  can  be  taken  over  just  those  y  l£  domgK  such  that 
Ajy  ■  v. 


PROOF.  Only  the  first  formula  will  be  proved,  as  the  second  can  be 
proved  similarly.  Let  J  denote  the  lower  conjugate  of  TT*A*.  The  defini¬ 
tions  yield  ^ 

J(u,v)  •  sup{<v*,v>  ♦  1nf{<u*,u>  -(£*(•, Aiv*) A*) (u*)}).  ' 

v*  u*  c 

Since  K*  Is  concave-closed.  It  follows  from  (34.3)  and  (6.3.1)  that 

r1(dom  £*(•  ,y*))  equals  r1(dom^K#)  when  y*  *  doir^K*  and  equals  R1” 

when  y*  i  don^K*.  Hence  (16.3)  and  the  hypothesis  range  Af  nrHdom^K*)  f  0 

Imply  (!<*(•  »AJv*)A|)*(u)  *  (A|7*(-(/^v*)*)(tt)  *  sup{k(x,A|v*) jA^x  »  u) 

for  every  v*.  where  k  denotes  the  concave  relative  of  K*.  Thus, 

J(u,v)  ■  sup{<v*,v>  ♦  sup,  k(x,A$v*)} 
v*  xcA1 *u  l 

•  sugj  sup{<v*,v>  -  {-k)(x,A|v*)>. 
xeAj  u  v* 

But  Theorem  0.2  Implies  -k  is  the  convex  parent  f  of  K,  and  hence  (16.3) 

implies  sup{<v*,v>  -  (-k)(x,  AJv*)}  »  (f(x,*)A|)*(v)  •  cl(A2f(x,*)*)(v)  - 
v* 

cl ( AgiK x,-))(v).  This  establishes  the  asserted  formula  for  J.  Finally, 
for  each  x  t  dom^K,  the  fact  that  K  Is  convex-closed  Implies  K(x,<)  and 
hence  cltAgKfx,*))  Is  constantly 

THEOREM  2.8.  Let  K  and  A  be  as  In  Lemma  2.7  and  assume 


range  A*r\r1(dom  K*)  f  i.  Define  functions  and  J2  on  RH  x  Rs  by 

J,(u,v)  *  sup  Inf  K(x,y) 

<x|  A-|X  •  u)  {y|Agy  -  v) 


J2(u,v)  ■  Inf  sup  K(x,y). 

{yjAgy  *  v)  (x|AjX  «  u) 

Then  there  exists  an  equivalence  class  CAK]  which  contains  every  concave- 
convex  function  J  on_  Rp  x  Rq  satisfying  <_  J  <  J2.  Moreover,  CAK] 

Is  closed  and  proper  and  Its  conjugate  Is  CK*A*].  If  [K]  Is  polyhedral, 
then  CAK]  1s_  polyhedral . 


PROOF.  Theorem  2.2  Implies  that  K*A*  and  7*A*  belong  to  a  closed 
proper  equivalence  class  [K*A*].  Let  [AK]  denote  the  conjugate  equiva¬ 
lence  class.  From  Lemma  2.7  and  the  fact  that  cl  f  <,  f  for  any  convex 
function  f  and  cl  g  >,  g  for  any  concave  function  g,  it  follows  that 

flftA*)*  <  Jj  and  J2  <  (k*A¥)* 

Hence  Theorem  0.1(b)  Implies  that  each  concave-convex  function  0  on 
Rp  x  Rq  satisfying  i  J2  belon9s  to  CAK].  The  polyhedral  asser¬ 

tion  follows  from  Corollary  2.4.1  and  the  fact  that  K*  Is  polyhedral 
whenever  K  Is. 

The  following  lemma  dualizes  the  hypothesis  used  throughout  this  sec¬ 
tion. 

LEMMA  2.9.  Let  K  and  A  be  as  In  Lemma  2.7.  Then  for  J  ■  1  and  2 
the  following  conditions  are  equivalent: 

( aj)  range  AJA  r1(domjK*)  t  0; 

(bj)  Ajl{0)  A(rec  conejK)  Is  a  subspace; 

(Cj)  Ajl{0>n(rec  cone^K)  C.  -(rec  conejK). 

PROOF.  Apply  Lemma  0.5  with  Lj  ■  range 

We  conclude  this  section  with  two  examples  showing  that  Theorems  2.2 
and  2.8  can  fall  If  their  hypotheses  are  weakened.  These  examples  are  pre¬ 
sented  In  the  notatlonal  scheme  of  Theorems  1.1  and  2.2. 


EXAMPLE  2.10.  Take  m  ■  n  «  p  ■  q  ■  1,  and  let  Aj  and  Ag  each  be 
the  zero  transformation  on  R.  Let  K  be  a  member  of  the  equivalence  class 
of  closed  proper  concave -convex  functions  on  R  x  R  having  as  kernel  the 
function 

(u»v)  u\  V(u,v)  C  (0,1)  x  (0,1). 

(This  equivalence  class  Is  discussed  in  [44,  p.  360].)  Then  dom  K  ■ 

[0,1]  x  [0,1],  K(0,0)  -  0,  £(0,0)  •  1,  and  K(u,v)  ■  uv  ■  £(u,v)  whenever 


(u,v)  c  dom  K\{(0,0)>.  Moreover,  for  each  a  c  [0,1]  the  function  Ko 
belongs  to  [K],  where  K  (0,0)  ■  a  and  K  (u,v)  ■  K(u,v)  whenever 
(u,v)  f  (0,0).  Observe  that,  for  J  ■  1  and  2,  range  Aj  AdomjK  t  0  while 
range  Aj  Arl(domjK)  ■  0.  Also,  for  any  K  c  [K],  the  function  KA  Is 
constantly  equal  to  £(0,0).  Since  0<.£(0,0)  <  1,  this  Implies  that  Ra 
Is  closed  and  proper.  However,  it  also  Implies  that,  for  any  two  elements 
Kj  and  K2  of  [K],  K,A  Is  equivalent  to  KgA  Iff  Kj(O.O)  •  K2(0,0). 
Thus,  as  £  ranges  over  [K]  the  functions  £a  determine  2^  distinct 
equivalence  classes  of  closed  proper  saddle-functions  (cf.  Theorem  2.2). 

Now  let  Jj  and  J2  be  as  In  Theorem  1.1.  Since  AJ  Is  the  zero  trans¬ 
formation  on  R, 

/-sup  Inf  K*  If  u*  -  0  and  v*  ■  0 
R  R 

Jj(u*,v*)  *  +*  If  u*  -  0  and  v*  f  0 

if  u*  t  0 


and 

/'Inf  sup  IT*  If  u*  *  0  and  v*  *  0 
1  R  R 

^(u^.v*)  *  <  —  If  it*  M  and  v*  »  0 

^  4-  If  v*  1 0. 

But  sup  Inf  K*  ■  -(£*)*(0,0)  ■  -£(0,0)  ■  -1,  and  similarly  Inf  sup  £-  *  0. 

R  R  “  R  R 

Hence  and  J2  are  closed  and  proper  but  not  equivalent  (cf.  Theorem  2.8). 

EXAMPLE  2.11.  Let  K  and  Aj  be  as  In  Example  2.10,  but  now  let  Ag 

be  the  Identity  transformation  on  R.  Observe  that  range  Ag  Ar1(dom2K)  t  0 

and  range  A^  A  dom^K  i  0  but  range  Aj  A  ri(dom^K)  ■  0.  For  each  £  c  [K], 

rO  If  v  c  (0,13 

KA(u,v)  «  £(0,v)  ■  £(0,0)  if  v  °  0 

'  4-  If  v  i  [0,1], 

where  0  <.£(0,0)  <.  1.  This  Implies  that  KA  is  proper  with  domain 


R  *  [0,1]  and  that  &  Is  closed  Iff  K(0,0)  ■  0.  It  also  implies  that, 
for  any  two  elements  Kj  and  Kg  of  [K],  K.A  Is  equivalent  to  KgA  Iff 
Kj(0,0)  *  Kg(0,0).  Recalling  the  functions  Ka  for  <x  e  C0,1],  we  conclude 
that  as  K  ranges  over  [K]  the  functions  KA  determine  2  *  distinct 
equivalence  classes  of  proper  saddle-functions,  where  only  the  class  con¬ 
taining  KA  is  closed  (cf.  Theorem  2.2). 


§3:  Sharper  Results 


In  this  section  Theorem  1.1,  which  has  already  been  sharpened  in  |2,  Is 
sharpened  still  further.  The  principal  results  are  Theorems  3.4  and  3.5. 
Among  the  conclusions  are  facts  concerning  the  attainment  of  the  extrema 
appearing  in  the  definitions  of  Jj  and  J2.  lemma  3.6  states  simple  con¬ 
ditions  sufficient  for  the  more  general  hypotheses  of  Theorems  3.4  and  3.5 
to  hold. 

Throughout  !3  we  adopt  the  notational  setting  of  Theorem  2.8.  That  is, 

K  is  a  closed  proper  concave-convex  function  on  r"1  x  Rn,  A  *  x  Ag  Is 

a  linear  transformation  from  R^  x  Rn  to  Rp  x  Rq,  and  Jj  and  Jg  are 

functions  defined  on  Rp  x  Rq  by 

Ji(u,v)  =  sup  Inf  K(x,y) 

{ x | A-| x  »  u)  iy|A2y  *  v) 

and 

J2(u,v)  *  inf  sup  E’fx.y). 

{ylAjy  »  v>  (x | x  ■  u) 

LEMMA  3.1 .  Let  f  be  a  proper  convex  function  on  Rn,  Ut  0  be 
a  convex  set  such  that  0  C  dom  f  C  cl  0,  and  let  E  be  a  convex  set  such 
that  E  r\  rl  0  f  i.  Then 

inf  f  ■  inf  f. 

CAO  E 

PROOF.  By  (6.3.1),  DC  dom  feel  D  implies  ri  0  a  ri^dom  f).  Hence 

S  a  E  r\  ri(dom  f)  cr  E  D  C  E  implies  trivially  that 

inf  f  >  Inf  f  >  inf  f. 

S  ”EHD  “  E 

Let  y  c  E  be  given,  if  y  i  dom  f,  then  f ( y)  ■  +•  >  inf  f.  Suppose 

“  S 

y  e  dom  f.  Since  EHri  0  t  d.  we  can  pick  an  x  c  S.  Then  (6.1)  Inpl  1  e*i 

that  yx  ■  (1  -  A)x  +  Ay  t  S .  for  each  0  5  A  <  1.  Hence  (7.5)  implies  that 

f(y)  >  cl  f(y)  3  11m  f(y  )  >  inf{f(y  )|0  <  A  <  1}  >  inf  f.  This  shows  that 
All  A  *  S 

f(y)  >  Inf  f  for  every  y  c  E.  Thus  inf  f  >  inf  f,  and  the  proof  is 
“  S  E  ”  S 


complete. 

THEOREM  3.2.  (u,v)  t  A  r1(dom  K)  and  assume  thjt 

Dirtc  cone  K(x,*)|x  c  rl(domjK),  A^x  ■  u} 
and 

Ajl(0)riO{rec  cone  K(*,y)|y  «  r1(dom2K),  A^y  ■  v) 
are  subspaces.  Then  there  exists  a  nonempty  closed  convex  product  set 
X  x  y  In  dom  aK  A  A“l{(u,v))  such  that  (x,y)  *  X  *  Y  Iff  (x,y)  .Is.  a 
saddle-point  of  K  with  respect  to  A'^u.v)}  for  each  Kc  [K].  If  the 
two  sets  In  the  hypothesis  are  actually  null  spaces,  then  X  *  Y  jjs  bounded. 

PROOF.  Define  a  concave-convex  function  L  on  Rf”  x  Rn  by 


0  If  A|X  ■  u  and  Agy  ■  v 

t(x,y)  *  j  +-  If  AjX  ■  u  and  Agy  f  v 
-  If  AjX  t  u 

Clearly,  L  Is  closed  and  has  domain  A'^C^v)}.  Since  (u,v)  c  A  r((dom  K), 
r1(dom  K)Ari(dom  L)  f  0.  Therefore  Theorem  4.2  (which  doesn't  depend  on 
the  results  of  !3)  Implies  that  the  equivalence  class  CK]  +  [L]  Is  defined 
and  has  domain  S  *T,  where  S  ■  Aj^uJHdomjK  and  T  ■  A2l{v)  AdomgK. 
Moreover,  Theorem  4.2  also  Implies  that  for  any  I?  e  [K],  [K]  +  [L]  contains 
the  closed  proper  saddle-function  M  given  by 

(  K(x,y)  If  x  c  S  and  y  e  T 

M(x,y)  -  \  *•  If  x  e  S  and  y  d  T 

^  -  If  x  t  S 

Suppose  x  e  rl  S  °  Ajl{u}/^\ rl(dom^K)  (use  [6.5]).  Then  (34.3)  Implies 
K(x , • )  ■  K(x,*)  Is  a  closed  proper  convex  function  with  effective  domain 
dom2K.  Hence  (9.3)  and  the  definition  of  M  Imply 

rec  M(x,«)  •  rec  K(x,*)  +  rec  «(*|A21{v)). 

But  rec  $(•  | A^ 1 { v } )  •  *(• |A^l{0)).  Therefore 
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re c  M(x,*)  ■  A^IOJHrec  cone  K(x,*)* 

Since  M(x,*)  «M(x,*)  whenever  x  c  rl  S  (Theorem  0.1(b)),  this  Implies 
that 

rec  conegM  ■  A^tOJA/^frec  cone  K(x,»)|x  c  rl  S). 

By  hypothesis  this  1$  a  subspace.  Similarly,  rec  cone^M  Is  a  subspace. 

It  follows  from  Lemma  0.4  that  (0,0)  e  r1(dom  M*),  and  hence  (37,5.3)  1m- 
plies  that  3M*(0,0)  Is  a  nonempty  closed  convex  product  set  X  *  Y.  By 
(37.5),  3M*(0,0)C  dom  3M.  But  Theorem  4.9  (which  doesn't  depend  on  the 
results  of  f3)  Implies  that  dom  3M  »  dom  3K  Adorn  3L,  and  (37.4)  Implies 
dom  SLCdom  L.  Therefore  X  *  Y  Cdom  3K  AA“1{(u,v)).  Now  (x,y)  t  X*  I 
Iff  (x,y)  is  a  saddle-point  of  M,  which  (by  (36.3))  occurs  Iff  (x,y)  Is 
a  saddle-point  of  K  with  respect  to  S  n  T.  Using  (x,y)  e  dom  K  together 
with  (34.3)  and  Lenina  3.1,  It  follows  that  this  occurs  Iff  (x,y)  Is  a 
saddle-point  of  K  with  respect  to  A*  {(u,v)h  Since  any  member  of  [K] 
could  be  taken  In  the  definition  of  M,  (36.4)  Implies  that  (x,y)  t  X  *  Y 
Iff  (x.y)  Is  a  saddle-point  of  ft  with  respect  to  A“l{(u,v) >  for  each 
ft  e  [Kj.  Finally,  suppose  the  sets  In  the  hypothesis  are  actually  nullspaces. 
Then  rec  cone^N  ■  {0}  for  j  ■  1  and  2,  so  that  Lemma  0.4  Implies 
(0,0)  c  1nt(dom  M* )  From  this,  (34.3)  and  (23.4)  It  follows  that  the  sets 
3M*( • ,0) (0)  «  X  and  3M*(0,«)(0)  ■  Y  are  bounded. 

LEMMA  3.3.  For  x  e  dom^K  the  following  three  conditions  are  equiva¬ 
lent,  and  they  Imply  A^x  c  dom^J^: 

(aj)  range  AJ  A rX  dom  K(x,* )*);.* 

(a2)  A^ 1  ( 0 >  n rec  cone  K(x,*)  Is  a  subspace; 

(a^)  A^IOJ/Arec  coneK(x,*)C  -rec  cone  K ( x , • ) . 

Similarly,  for  y  e  domgK  the  following  three  conditions  are  equivalent 
and  they  Imply  Kp  e  don^Jg1 
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(bj)  range  A}  f\  r1(dom  K(*,y)*);- 

(b2)  Ajl(0)n  rec  cone  JT(*  »y)  is^  a  subspace; 

(b3)  Aj^OJArec  cone  ff(*,y)C  -rec  cone  F(*,y). 

PROOF.  Only  the  first  assertion  Is  proved,  as  the  second  can  be  proved 
similarly.  Since  rec  cone  JC ( x , • )  Is  a  convex  cone,  clearly  (a^)  holds 
Iff  (a3)  holds.  By  Theorem  0.1,  K(x,0*  Is  proper  convex  and  its  conju¬ 
gate  is  K(x,*)«  Hence  (16.2.1)  Implies  that  (a^)  falls  Iff  (a3)  fails. 

Thus,  the  three  conditions  (a-|)  -  (a3)  are  equivalent.  Suppose  now  that  x 
satisfies  (a3).  Since  K(x,*)  Is  closed  proper  convex,  (9.2)  implies  that 
AgKfx#*)  is  too.  Hence  A2<K(x,*)  Is  never  — .  But  A2K(x,*)  <.  J^A^x,*). 
Therefore  A^x  c  dorojjj. 

THEOREM  3.4.  Assume  that  each  x  e  ri(dom^K)  (resp.  y  t  rl(dom2K)) 
satisfies  one  of  the  equivalent  conditions  (a^)  (resp.  (b.))  of  Lemma  3.3. 
Then  the  conclusions  of  Theorem  2.8  hold,  and 

ri(A  dom  K)  C  dom  AK  C  A  dom  K. 

Furthermore,  for  each  (u,v)  c  r1(A  dom  K)  there  exists  &  nonempty  closed 
convex  product  set  X*Y  In  dom  aK/^i  A~1((u,v))  such  that  (1)  (x,y)  c 
X  x  Y  j_ff  (x,y)  is  a  saddle-point  of  K  with  respect  to  A“ 1  { ( u , v) ) 
for  each  %  t  [K],  and  (2)  (x,y)  c  X  x  Y  Implies  J(u,v)  *  K(x,y)  for 
every  J  c  [AK]  and  K  r  [K], 

PROOF.  The  hypothesis  Implies  that  Aj1 {0> A  rec  conejK  Is  a  subspace 
for  j  »  1  and  2.  Hence  by  Lemma  2.9  the  conclusions  of  Theorem  2.8  hold, 
and  In  particular  and  02  belong  to  [AK],  where  [(AK)*]  *  [K*A*J. 

Thus  dom  AK  a  dom^J^  x  dom2J2.  Therefore  the  hypothesis  and  Lenina  3.3 
Imply  that  ri ( A  dom  K)  Cl  dom  AK.  On  the  other  hand.  Lemma  2.9  and  Corol¬ 
lary  2.4.2  Imply  that  dom  AK  C  A  dom  K.  Let  (u,v)  c  ri(A  dom  K).  3y 
Theorem  3.2  there  exists  a  nonempty  closed  convex  product  set  X  x  y  in 
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dom  3KO  A’l{(u,v) }  such  that  (1)  holds.  Suppose  (x,y)  e  X  *  Y.  Since 
(1)  Implies  (x,y)  Is  a  saddle-point  of  K  with  respect  to  A“ 1 { ( u ,v) } , 

certainly  J^(u,v)  *  K(x,y).  Since  r1(A  dom  K)  *  r1(dom  AK)  by  (6.3.1), 
Theorem  0.1(b)  Implies  that  Jj(u,v)  ■  7(u,v)  for  each  7  e  [AK],  Also, 
(x,y)  c  dom  3K  and  (37.4.1)  imply  that  K(x,y)  *  T<(x#y)  for  each  K  e  [k]. 
This  establishes  (2). 

THEOREM  3.5.  Assume  that  each  x  c  dom^K  (resp.  y  e  dom2K)  satisfies 
one  of  the  equivalent  conditions  (a^)  (resp.  (b^))  of  lemma  3.3.  Then  dom  AK 
actually  equals  A  dom  K.  Moreover,  writing  cl2(AK)  *  0  and  cl ^ ( AK)  T, 

*  J  and  Jg  *  ^  ~  r«nge  A. 

In  particular.  J^u.v)  •  J(u,v)  except  when  uc  range  A^  \  A^dom^K  and 
v  i  range  Ag,  and  J2(u,v)  ■  D(u,v)  except  when  u  t  range  A^  aiid 
v  e  range  A^\  A2dom2K. 

PROOF.  By  Lemma  3.3,  A  dom  KCTdom^J^  x  dom2Jg.  From  this  it  follows 
as  In  the  proof  of  Theorem  3.4  that  A  dom  K  ■  dom  AK.  We  only  prove  the 
assertion  about  Jj,  as  the  other  is  similar.  From  the  proof  of  Lemma  3.3, 
A,,K(x,*)  Is  closed  for  each  x  t  dom^K.  Hence  Lemma  2.7  Implies 

J(u,v)  ■  sup{A2K(x,‘)(v)[X  e  dom^K,  A^x  «  u).  (1) 

If  x  i  dom^K,  then  JK ( x , • )  Is  constantly  -*»,  so  that 

C  —  If  it  range  A, 

A-KU.'Hv)  ■  {  (2) 

t  +•  if  v  i  range  A,,. 

Since  Jj(u,v)  «  sup(A2K(x,«)(v)|A1x  *  u}  by  definition,  (1)  implies  that 
Jj(u,v)  equals 

max(J(u,v),  supfAgKfx.'MvJlAjX  »  u,  x  i  dom^KH, 
which  by  (2)  equals  J(u,v)  whenever  v  c  range  Ag.  Henceforth  assume 
v  l  range  A2.  Suppose  u  c  A^om^K.  Pick  an  x  e  dom^K  such  that  A j/  *  u. 
Since  Ag 1  { v )  ■  4,  +»  ■  A2K(x,*)(v)  <  J(u,v)  <  J|(u,v).  Hence 
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J(u,v)  *  Jj(u,v)  ■  *•  whenever  u  e  A|domjK.  Observe  also  that  the 
convention  sup  t  *  -«  implies  J(u,v)  _<  J^(u,v)  ■  —  whenever  u  l  range  Aj . 
In  the  only  remaining  case,  l.e.  when  u  c  range  A^\.  A^om^K,  (1)  Implies 
J(u,v)  ■  sup  4  ■  —  while  J^(u,v)  ■  supdnf  0 1 A-|X  ■  u)  *  +~. 

While  the  hypotheses  of  Theorems  3.4  and  3.5  are  general,  they  may  ap¬ 
pear  somewhat  cumbersome  to  check.  The  next  lemma  gives  simpler,  "global11 
conditions  on  K  and  A  which  Imply  the  hypotheses  of  both  Theorems  3.4 
and  3.5. 

For  a  nonempty  convex  set  C  In  Rn,  define  the  recession  cone  of  C 
to  be  the  set 

0+  C  •  (y|x  +  Ay  c  C.Vx  e  C  V  A  >  0) . 

LEMMA  3.6.  The  three  following  conditions  are  equivalent,  and  they 
Imply  that  conditions  (a^)  -  (a3)  of  Lemma  2.-1  hold  for  each  x  e  dom^K: 

(Cj)  A^IOinO*  cl(dom2K)  »  {0}, 

(c2)  A^Ivl^domgK  is  bounded  for  each  v  e  R^; 

( c3)  Agl  { v)  n  rl  {domgK )  .1$,  nonempty  and  bounded 
for  some  v  e  Rp. 

Similarly,  the  three  following  conditions  are  equivalent,  and  they 
Imply  that  conditions  (b^)  -  (b3)  of  Lemma  3.3  hold  for  each  y  e  dom^K: 

( d| )  Ajl{0ino+  cKdom^)  *  {0}; 

( dg)  A| 1  { u}  A  dom^K  is  bounded  for  each  uc  Rp; 

(d3)  Aj^u)  C\  ri  (dom^K)  Is  nonempty  and  bounded 
for  some  u  e  Rp, 

PROOF.  Only  the  first  assertion  is  proved,  since  the  second  is  similar. 
For  each  v  c  A^omgK,  (8.3.3)  and  (8.4)  imply  that 

A^lvincl  (dom2K)  Is  bounded  Iff  A^IOlAO*  cl  (doiy.)  =  10).  (') 

It  follows  from  this  that  (Cj)  Implies  (c2).  By  picking  any  v  e  A^ri  (dori?K) 


It  follows  that  (Cg)  Implies  (c^).  Now  assume  (c^).  Then  (6.3.1)  and 
(6.5.1)  Imply  that 

A^1  ( v>  AcUdon^K)  ■  A^Cv}  n  c1(r1(dom2K))  ■  cl(A^{v}nr1(dom2K)). 

That  this  set  Is  bounded  follows  from  the  fact  that  Ag1  C v)  nr1(dom2X) 

Is  bounded.  Hence  (cj)  follows  by  (1).  Finally,  let  x  c  dom^K  be  given. 
Write  f  *  K(x,»)  and  C  ■  dom  f.  Then  by  (34.3)  and  (6.3.1),  f  Is  a 
proper  convex  function  with  cl  C  -  cl(dom2K)  But  by  (8.5)  and  (8.1)  It 
follows  easily  that  dom(rec  f)cz  0+  C  C  0+(cl  C).  Hence 
rec  cone  K(x,*)  Cl  0+(cl  dom2K),  and  therefore  (c^)  Implies  that  x  satis¬ 
fies  (a2)  of  Lemma  3.3. 

Finally,  observe  that  If  conditions  (c^)  and  (dj)  of  Lemma  3.6  are  met, 
then  the  sets  X  and  Y  given  by  Theorem  3.4  for  each  (u,v)  c  A  rl (dom  K) 
are  actually  bounded  and  hence  compact.  This  Is  because  the  two  sets  In  the 
hypothesis  of  Theorem  3.2  are  then  nullspaces. 


§4:  Addition  and  Minimax  Convolution 


This  section  begins  with  the  development  of  the  addition  operation  on 
equivalence  classes  of  saddle-functions.  Next,  some  results  concerning  separ¬ 
able  saddle-functions  are  presented.  These  are  used  together  with  the  results 
of  i<2  and  3  to  define  another  operation  on  equivalence  classes  of  saddle- 
functions.  This  operation,  called  minimax  convolution.  Is  dual  to  addition. 
The  theorems  proved  concerning  these  dual  operations  closely  parallel  existing 
theorems  about  the  dual  operations  of  addition  and  Inflmal  convolution  on 
convex  functions. 

There  are  two  technical  difficulties  involved  In  defining  the  operation 
of  addition.  The  first  stems  from  the  fact  that  we  are  working  with  extended- 
real-valued  functions;  we  must  deal  somehow  with  the  expression  *•  -  «.  The 
second  and  more  fundamental  difficulty  Is  that,  from  the  point  of  view  of 
minimax  theory,  we  want  to  define  addition  of  whole  equivalence  classes  and 
not  just  individual  functions.  The  following  definition  is  designed  to  handle 
both  these  difficulties. 

For  1  »  1,...,s  let  Kj  be  a  concave-convex  function  on  r”5  x  Rn 
with  domain  x  o^.  We  say  that  +...+  [Ks]  Is  defined  iff  the  sets 
C  •  and  D  «  DjA...<*!Ds  are  nonempty  and 

Sfyx.y)  *  £ Kj(x,y),  V(x,y)  e  rl  C  x  rl  D 
whenever  ^  e  [K^]  ,...,  K$  e  [Ks].  In  this  event  [Kjj  +...+  [K$],  which 
will  usually  be  written  as  [K^  +...+  K$],  or  [2Kj],  Is  defined  to  be  the 
unique  equivalence  class  of  closed  proper  concave-convex  functions  on 
R?”  x  Rn  having  as  kernel  the  function  on  rl  C  x  ri  3  given  by  (x,y)  > 
SKj{x,y).  Such  a  unique  equivalence  class  exists  by  (34.5.1).  The  opera¬ 
tion  which  sends  [Kj]  . [K^]  into  [Kj  +...+  Ks3  is,  quite  naturally, 

called  addition. 

LEMMA  4.1.  For  1  =  l,...,s  let  be  a  closed  proper  concave-convex 
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function  on  fi"1  *  Rn  with  domain  Cj  *  Dj.  Then  [Kj]  ♦...+  [Ks3  fs 
defined  XL  either  Cj  A  . . .  A C-s  f  i  and  rl  Dj  C\ . . .  A  rl  D$  f  i  or 

ri  C  j  /'“'v . . .  r\  rl  Cf  t  i  end  Dj  C\ . . .  A  05  f  f. 

PROOF.  This  follows  eeslly  from  (6.5)  and  Theorem  0.1(b). 

It  1$  actually  not  hard  to  establish  a  weaker  condition  sufficient  for 
[Kj]  +...+  CKg]  to  be  defined.  Loosely  speaking,  the  condition  Is  just 
that  the  Kj  be  closed  and  that  (possibly  after  renumbering  the  K^'s) 
there  exist  an  Integer  r,  0  <.  r  <  s,  such  that 

rl  Cj  n  . . .  A  rl  Cr/^  f\ . . .  C\  C$  f  i 
and 

Dj  f\ . . .  C\  Dp  A  rl  n...Ar1  Ds  t  0. 

(The  conditions  In  Lemma  4.1  correspond  to  the  values  r  -  0  and  r  ■  s  -  1.) 

Instead  of  appealing  to  (6.5),  the  proof  uses  the  generalization  of  (6.5) 
described  In  the  Appendix. 

THEOREM  4.2.  Let  Kj,...,K$  be  closed  proper  concave-convex  functions 
on  R*"  x  Rn  such  that  r1(dom  Kj)fW..nr1(dom  Kg)  t  d.  Then  [Kj]  +. . .+  [K$] 
is  defined ,  has  domain  dom  Kj  C\ . . .  C\ dom  K$ ,  and  contains  the  closed  proper 
saddle-function  K  given  by 

/XKj(x,y)  jf.  x  e  C  and  y  e  D 

K(x,y)  »<  *»  If  x  e  C  and  y  i  0 

v  -  if  x  i  C 

PROOF.  Lemma  4.1  implies  [Kj]  +..,♦  [Ks3  is  defined.  Hence  It  Is  the 

unique  equivalence  class  of  closed  proper  concave-convex  functions  on  R™  x  R° 
having  the  same  kernel  as  K.  Therefore  by  (34.4)  the  proof  will  be  complete 
once  we  show  K  Is  closed.  This  we  do  by  checking  that  K  satisfies  the 
six  conditions  of  (34.3).  This  follows  routinely  by  applying  (34.3)  to  the 
Kj's  with  the  aid  of  (6.5). 
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In  order  to  apply  the  results  of  H2  and  3  to  an  equivalence  class 
CKj  +...+  K$]  and  Its  conjugate,  we  need  to  define  and  establish  some  pro¬ 
perties  of  "separable”  saddle-functions.  For  i  ■  s  let  K.  be  a 
proper  concave-convex  function  on  R^i  *  Rn1  with  domain  x  o. .  Write 
m  *  n  =  Xn^  ancf  define  a  function  (K, ,...K5)  on  Rm  *  by 

if  x  e  C  and  y  e  D 
(Kj , . . .  ,K$)  •  <  +•  i f  x  r.  C  and  y  i  D 

I*  -  if  xH 

where  x  -  (x1,...,xs),  y  ■  (y1 ,...,ys),  C  *  Cj  x...x  c$>  0  *  D1  .*  D$. 
With  the  aid  of  (34.3)  and  the  following  Lemma  4.3,  it  can  easily  be  verified 
that  the  function  (Kj,...,Ks)  Is  concave-concave  with  domain  C  *  C.  Such 
a  saddle-function  Is  called  separable.  Lemma  4.3  reviews  some  useful  pro¬ 
perties  of  separable  convex  functions,  and  In  Theorem  4.4  similar  properties 
are  established  for  separable  saddle-functions. 

LEMMA  4.3.  For  1  =  1 . s  Jet  f^  be.  a  proper  convex  function  on 

Rni  with  effective  domain  C^.  Define  C  *  x...x  C$  arjd  f(x^,...,xs)  = 

f-j(xj)  +...+  f$(xs).  Then  the  following  statements  hold: 

(a)  f  is  proper  convex  with  effective  domain  C; 

(b)  (cl  f)(x1,...,xs)  ■  (cl  fjMx^  +...♦  (cl  fs)(xs); 

(c)  f  is  polyhedral  if  each  f.  Is.; 

(d)  f*(xf,...,x*)  -  ff(xf)  +...+  fj(x*); 

(e)  3f(x1 . x$)  *  afj(xj)  x...x  3fs(x$); 

(f)  (rec  f)(yr...,ys)  •  (rec  f^fy^)  *...+  (rec  f$)(ys). 

PROOF.  Assertions  (a)  and  (d)  are  trivial.  Assertion  (1)  follows  imme¬ 
diately  from  (a)  and  (8.5).  To  see  (b),  let  x  -  (xj,...,x  )  t  cl  C  s 

cl  x...x  cl  C$  be  given  and  fix  any  x°  *  (xj,...,x°)  e  rl  C  * 

ri  C1  x...x  ri  C$.  Define  x^  s  (x^,...,x*)  by  x^  *  (1  -  a)xq  +  Ax  for 


0  <  x  <  1.  Then  (a)  and  (7.5)  Imply  that 

(cl  f)(x)  3  11m  f(x.)  «  Ellm  f.(x})  «  Z(cl  fj(x.). 

Atl  A  A+l  1  1  11 

On  the  other  hand,  If  x  t  cl  C  then  Xj  i  cl  Cj  for  some  1  <.  j  <.  s  and 

hence  (a)  and  (7.4)  Imply  that  (cl  f)(x)  »  +*  •  (cl  fj)(Xj)  iX(cl  f^Mx^). 

This  proves  (b).  To  see  (c),  define  hj(xj, . x  )  »  fj(x^)  tor  each  *  • 

Then 

epl  h1  *  {(x1,...,xs,p)|(xi,u)  c  epl  f^ 
and  epl  f^  polyhedral  Imply  that  epl  h^  Is  polyhedral  for  each  1.  Hence 
(19.4)  Implies  that  f  ■  hj  ♦...+  h$  Is  polyhedral.  Finally,  we  prove  (e). 
Suppose  first  that  x  «  (x,,...,x  )  l  C.  Then  (a)  and  (23.4)  imply  that 
af(x)  =  t  and  also  3fj(xj)  e  i  for  some  1  <  j  <  s.  Thus 
3f.j(xj)  +. ..+  3 f $ ( x$ )  »  t.  Now  suppose  that  x  e  C.  Using  (6.1)  and  (7.5), 
one  can  easily  verify  that,  for  a  convex  function  h  on  Rn  and  a  subset  C 
of  Rn  containing  ri(dom  h),  x*  c  3h(x)  Iff 

h(y)  >  h(x)  +  <x*,y  -  x>,  Vy  c  C. 

Applied  to  the  situation  at  hand,  this  implies  that  x*  ■  (x|f,...x*)  c  5 f ( x) 


^2f1(y1)  >  Z(fi(xi)  +  <xf,y^  -  xt>)  (1) 

for  every  (y^,...,y$)  e  C.  Let  j  be  any  fixed  Index.  By  letting  y^ 
vary  over  Cj  and  requiring  y^  *  x^  for  i  f  j,  (1)  implies 

W  +  ^3  W  i  W  *  <x]^j  ■  Y  +  ^,(h<x()  ♦  <x*-xi  -  V>- 

Since  all  the  numbers  f^(x^)  are  finite,  this  reduces  to 

W  1  W  +  <Xj,yj  '  Y*  e  Cj‘  (2) 

But  this  is  equivalent  to  x*  *  3fj(xj).  Thus  we  have  shown  that  (1)  implies 
xj  e  3 f j ( x j )  f°r  J  *  The  converse  follows  easily  by  summing  the 

s  inequalities  of  the  form  (2).  This  completes  the  proof  of  (e). 
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THEOREM  4.4.  for  i  *  1  fet  Kj  be  a  closed  proper  concave- 

convex  function  on  1^1  x  Rn1  with  domain  C|  k  D|,  Put  K  -  (Kj,...,K  ) 
and  write  C  *  x...x  Cs,  0  ■  x...x  0^,  x  3  (xj,...,x  )  and  y  = 

(y^*****ys)‘  Then  the  following  statements  hold: 

(a)  K  Is  closed  proper  concave-convex  with  domain  C  x  0. 

(b)  If  c  !.K^  j  .  for  i  -  l,...,sf  then  ( K  ^ ....  ,*K$ )  e  [K]  (i.e. 

[K]  depends  only  on  J).  .. 

( c)  The  least  and  greatest  members  of  [K]  are  given  by 

)  ]f  x  r  C  and  y  e  cl  D 

K(x,y)  *  4  *»  vf  x  e  C  and  y  4  cl  0 

i 

l  -  if  x  i  c 


and 


/'i.  K.. (x^  ,y^) 


K(x.y) 


if  x  e  cl  C  and  y  e  D 
If  x  t  cl  C  and  y  c  D 


v 


if 


y  t  D 


(d)  For  j  *  1  and  2  and  (x,y)  c  C  x  d, 

3jK(x,y)  »  3iK1(x1,y1)  *...*  ^(x^y,.) 

(and  3K(x,y)  *  d  whenever  (x,y)  <  C  *  D). 

(e)  (Kf,...K*)  c  [KM 

(f)  (rec1K)(x)  =  rec1K. )  ( x1 )  and  (rec2K)(y)  =  £(rec2Ki)(y1) 

(g)  II  T  ( resp.  f^)  denotes  the  convex  parent  of  K  (resp.  K.. ) , 
then  f(x,y*)  *  Hf.(x.  yt).  Similarly  for  concave  parents. 

(h)  If  each  Ki  is  polyhedral,  then  K  is  polyhedral. 

PROOF,  (a)  It  suffices  to  check  that  K  satisfies  the  six  conditions 
of  (34.3).  Let  x  s  C.  Then  x-  ?.  C-  together  with  (34.3)  applied  to  K^ 
imply  that  K^(x.  ,•)  is  a  proper  convex  function  with  effective  domain  con¬ 
taining  D. .  Since  K(x,y)  3  ^K^'x-.y^)  +  6(.yjD),  it  follows  from  lemma 


4.3(a)  and  (5.2)  that  K(x,»)  is  proper  convex  with  effective  domain  D. 

Now  suppose  x  t  ri  C.  Then  x^  t  ri  Cj,  so  that  (34.3)  implies  K(xj,*) 
is  closed  and  Its  effective  domain  actually  equals  Dj.  Thus  K(x,y)  * 
Ek^Xj^),  and  Lemma  4.3(b)  implies  K(x,*)  is  closed.  This  establishes 
the  first  two  conditions  of  (34.3)  for  K.  Of  the  remaining  four  conditions, 
two  have  parallel  proofs  and  the  other  two  are  satisfied  trivially. 

(b)  Let  K|  e  CK^3  for  1  ■  l,...,s  and  write  £  *  (K.| ,. . . ,K$) .  Since 
by  (34.4)  two  closed  proper  saddle-functions  are  equivalent  iff  they  have 

the  same  kernel,  ^ (x^ ,y^ )  •  K^(x^,y^)  whenever  (x^,y^)  e  ri  C.  *  ri  . 
Hence  K  and  K  agree  on  ri  C  *  ri  0.  Since  equivalent  saddle-functions 
have  the  same  domain,  dom  ?|  *  C|  *  D|,  This  implies  dom  K  -  C  x  d. 
Therefore  Sf  and  K  have  the  same  kernel . 

(c)  Since  K  Is  closed.  Theorem  0.1(b)  Implies  K  -  cl2K  and  K  =  cljK. 
If  y  i  0,  then  K(-,y)  equals  +•  on  rt  C  and  hence  K(*,y)  ■  cl(K(-,y))  ? 
*».  Now  suppose  y  e  0.  As  In  the  proof  of  part  (a),  K(x,y)  ■  SiKf(x^ty^)  - 
6(x (C)  Is  proper  concave  with  effective  domain  C.  Since  g(x)  «  Sk^x^,^) 
is  proper  concave  with  C  C  dom  g  c  cl  C  by  (34.3)  and  Lemma  4.3(a),  it 
follows  from  (6.3.1)  and  (7.3.4)  that  (cljK)(x,y)  •  (cl  g}(x).  But 

(cl  g)(x)  *  S,(cl|K^)(x.| ,y^)  by  Lemma  4.3(b).  Since  cljKj  -Kj.  this  es¬ 
tablishes  the  formula  for  K,  The  other  formula  is  proved  similarly. 

(d)  By  part  (a)  and  (37.4),  dom  3K  C  0*0.  Suppose  (x,y)  eC*0. 

By  (37.4.1),  3K(x,y)  **  3*(-,y)(x)  *  3K(x,-)(y).  Sut  from  part  (c)  and 
Lemma  4.3(e),  3K(*,y)(x)  3  3lC, ( ' .y1)(x1 )  *...*  3K$(* ,ys)(x$)  where  by 
(37.4.1)  the  Kj  can  be  replaced  by  Kj.  This  establishes  the  assertion 
for  j  *  1,  and  the  case  j  3  2  is  exactly  the  same. 

(e)  The  proof  is  by  induction.  First  observe  that  separable  saddle- 
functions  can  be  given  an  equivalent,  inductive  definition.  Namely,  for 
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s  *  2  let  the  definition  be  as  given  above,  and  for  s  i  define 
(Kj,...,Ks)  »  ((K^,...,KS-^),KS)  where  a  space  of  the  form 
(r”1!  x...x  Jfs-l)  *  iPs  is  tdenWfled  with  fPl  x...x  Rms.  For  the  pur¬ 
pose  of  this  proof  we  adopt  this  Inductive  definition.  Suppose  the  asser¬ 
tion  has  already  been  proved  for  the  case  s  =  2*  and  'let  s  >  2  be  fixed. 
Since  (Kf,...,K*)  *  ((K|, .. .  ,K*)  by  definition,  the  Inductive  hypo¬ 

thesis 

together  with  parts  (a)  and  (b)  Imply  that  (K^,...,K*)  is  equivalent  to 
((K, , K$ _ i ) * » K* ) .  But  by  the  case  s  *  2 

this  is  contained  in  T((K| . K^) >!<,.)*].  which  by  definition  is  the 

same  as  [(K, . .  >*,.)*].  Thus,  part  (e)  will  be  proved  once  the  case  s  *  2 
is  established.  *“St»  let  $  ■  2  and  write  dom  Kf  =  x  Of.  By  (36.3)  and 
(36.1), 

K*(x*,y*)  »  sup  inf  {£<x,  ,x?>  +  %,yf>  -  MXf.yJ} 
yeD  xeC  11  11 

1  sup  inf  {<x2,x£>  +  <y2,yj)>  ~  K2^x2,y2^  +  21^ x*»>r*) > 
ygeDp  X2V^?. 

/2K*(x|,,yJ)  if  xj  e  C|  and  yf  e  domKj(xf,-) 

=<+«  if  xf  e  C|  and  y^  i  dom  K|(x|,’) 

if  xf  i  Cf 

Moreover,  in  the  event  that  xf  e  CJ  and  c  dom  K|(x|,  )  we  have 
(££*(*(».*?)  c  R  if  x|  e  C*  and  e  dom  K*(xJ,  ) 
EK^xf.yf)  +«.  if  x£  e  C2  anci  y2  i  dom  KJfxf*  ^ 

if  xf  i  C| 

Also,  xf  e  Cf  implies  Of  d  dom  K|(x|,-)  by  (34.3).  If  C*  -  x  Cf, 
and  0*  =  D|  x  0|,  then  the  above  facts  imply 

dom^K*  C.'  C*.  0*  C  dom2K*. 
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and 

K*(x*,y*)  <  XKj'txf.yj')  whenever  x*  c  C*  or  y*  e  D*. 

Parallel  reasoning  starting  from  )T*(x*,y*)  yields  that 

C*  c  dom^*,  dom^C  0*. 
and 

21ty(xf,yJ)  <.  K*(x*,y*)  whenever  x*  t  C*  or  y*  e  0*. 

Therefore  dom  K*  *  C*  *  D*  and 

K*(x*,y*)  <  lKf(xf,y|)  <  lC*(x\y*) 

whenever  x*c'C*  or  y*  c  D*.  By  applying  Theorem  0.1(b)  to  K*,  It 
follows  that  (KJ,KJ)  and  K*  have  the  same  kernel.  Since  they  are  both 
closed  and  proper,  (34.4)  Implies  they  are  equivalent. 

(f)  From  the  definitions.  Theorem  4.4(a),  (34.3)  and  Lemma  4.3(f)  It 
follows  that 

(rec2K)(y)  ■  sup{(«*ec  K(x,*))(y)|x  e  rl  C) 

»  sup(£(rec  K)|(xi ,•  ))(y1> |X|  e  rl  Cj,...^  c  rl  Cs> 
«Zsup{(rec  K1(x1  ,•  ))(y1 ) |xf  e  rl  Cj) 

■  SKrec^Myj). 

The  other  formula  Is  proved  similarly. 

(g)  By  part  (c),  K(x,y)  *£Kf(Xj,yj)  whenever  x  c  C.  Hence  Theorem 

0.1(a)  Implies  that  f(x,y*)  ■  sup(<y,y*>  -  SjK^x^y^))  * 

£sup(<y^ ,y|>  -  K^(x^,y^)}  B  £f^(Xj,y|)  whenever  x  e  C.  On  the  other  hand, 
y1 

If  x  i  C  then  Theorem  0.1(a)  Implies  that  f(x,«)  and  fj(Xj.*)  for  some 
1  <  j  <.  s  are  constantly  Since  each  Ki  Is  proper,  each  f^  Is  proper 
and  hence  .never  Therefore  (x,y*)  ■+  Zf^(x^  ,yf)  Is  constantly  +» 
whenever  x  t  C.  This  establishes  the  formula. 

(h)  By  part  (g)  and  Lemma  4.3(c). 

For  the  remainder  of  54  let  certain  notation  remain  fixed  as  follows. 
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For  1*1 . s  let  K.  be  a  closed  proper  concave-convex,  function  on 

R"*  Rn  with  domain  *  0^.  Write  C  *  CjA...HCs  and  D  3  DjA..  ./V)  , 

and  define  K  *  ( K-| . K$).  Let  map  each  x  t  R01  into  the  s-tuple 

(x . x),  let  map  each  y  s  Rn  into  the  s-tuple  (y,...,y),  and 

put  A  *  Aj  *  A^. 

The  following  condition  is  frequently  used: 

ri(dom  K.jA. ../Ari(dom  K,.)  i  0  (•)£) 

The  next  lemma  dualizes  it.  v 

Lf  MMA  4.6.  The  condition  r  i  ( dorn^  )  C\ . . .  H  ri  ( dom, Ks )  {  <b  js.  equiva¬ 
lent  to 

5.'xf  »  0  and  rec^KJ)(xf)  >  0  ^  21(  rec^KJ)  ( -xj)  ^  0. 

Similarly,  tne  condition  ri  ( domett  j)  A.  .  .Ari(dom„KJ()  f  t>  n  eg  "i>!. a  .lent  .to 
Eyf  ~  d  and  Z(rec,K?)(yf)  <  0  2(rec?Kf)(-y?)  <  0. 

PROOF.  Apply  Lemma  O.b  to  tne  saddle-function  (Kf,...,K*)  and  the 
subspace  f (xf,. . . »x*) |xf  x*}  and  simplify  using  Theorem  4.4.  The 

second  assertion  is  proved  similarly. 

The  next  theorem  enables  us  to  apply  the  results  of  §52  and  3  to  the 
equivalence  class  [Kj  +  ...+  K^j. 

THEOREM  4.6.  Assume  (->4) .  Then  [Kj  +  ...+-  K ^ .)  is  defined  and  equals 

MOV). 

FROCr.  Theorem  4.2  implies  LKj  +...+  K.  ]  is  defined  and  has  kernel 
(x,y)  ►  ZK^x.y),  V(x,y)  t  ri  C  *  ri  0. 

Theorems  4.4(a)  and  2.2  imply  that  [KA1  exists,  and  it  is  easy  to  check 
that  its  kernel  is  the  function  given  above.  The  theorem  now  follows  from 
(34.4). 

COROLLARY  4.6.1.  Assume  Or).  J_f  each  [  K .  ]  is  polyhedral,  then 
I'Kj  +...*•  Ks J  is  polyhedral. 
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PROOF.  By  Theorem  4.4(h)  and  Corollary  2.4.1. 

COROLLARY  4.6.2.  Assume  (*).  ]±  h  (resfl.  k)  denotes  the  convex 
tea,  .CgafiiYfc)  fi&csrti  1 1  CK1+...4Ks3  M  fj  (jresp.  g{)  denote s  the 

jmen  tec.  cQQE&yfr)  msi&sL  ita 

b(x,y*)  «  infilf^(x,yf)|  2yf  *  y*> 

M 

k(x*,y)  ■  suplXg^xf.yJISxf  ■  x*h 
PROOF.  By  Theorems  2.4  and  4.4(g). 

COROLLARY  4.6.3.  Assume  (*).  Then 

dom  (K|  +...+  KS)*G  dom  KJ  +...+  dom  KJ. 

In  particular.  If  and  g^  are  as  In  Corollary  4.6.2.  then 

n(domj(kj  ■»...♦  K$)*)  »U{£n(dom  g^(*,y))|ye  ri  D) 
and 

ri ( donigC +...+  Ks)*)  *  U(Zr1(dom  f^(x»*))|x  e  ri  C) 
where  these,  formulas  also  hold  wi th  "ri "  deleted  throughout. 

PROOF.  Apply  Corollary  2.4.2,  using  Theorem  4.4(g)  and  Lemma  4.3(a)  to 
simplify. 

In  convex  function  theory  there  is  a  result  correspondin'*  to  the  inclu¬ 
sion  in  Corollary  4.6.3.  Namely, 

dom  (fj  +...+  t's)*  c  dom  f^  ♦...+  dom  f* 
whenever  are  proper  convex  functions  satisfying 

ri(dom  fj)fl.  ,.riri(dom  f$)  t  t>  ( see(  16.4) ) .  One  might  hope  in  the  saddle- 
function  case  to  have  at  least 

ri(dom  Kf  +...+  dom  KJ)  C  dom  (K1  +  ...+  Ks)*  C  dom  KJ  +...+  dom  K* 
satisfied  whenever  K^,...,KS  are  closed  proper  concave-convex  functions 
satisfying  ri  dom  K^O...riri  dom  K$  f  it.  However  this  can  fail  drastically, 
as  can  be  seen  by  taking  s  c  2  and  putting  Kj(x,y)  *  <x,y>  and 
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Kg(x,y)  *  -<x,y>  on  Rm  *  R*".  In  this  case  dom  (Kj  >  Kg)*  *  (0)  *  {0} 
whereas  dom  Kf  *  r"1  *  iF  8  dom  Kg.  Theorems  4.8  and  4,11  give  conditions 
which  guarantee  that  this  "collapsing"  behavior  of  dom  (K.^  +...*  K$)* 
cannot  occur. 

LEMMA  4.7.  Assume  (*).  Ihea 

(rec^  +...♦  K$))(x)  ■  1nf(2L(rec  K^(» ,y))(x)|y  e  rl  D) 
and 

(rec2(K|  ♦  ...+  K$))(y)  «  sup{£(rec  Kj(x,* ))(y)|x  e  rl  C). 

PROOF.  By  Theorem  4.6  the  formulas  In  Lemma  2.5  can  be  applied.  Sim¬ 
plify  using  (34.3)  and  Lemma  4.3(f). 

THEOREM  4.8.  Assume  (*).  Ihen  for  J  »  1  and  2, 

cl(domj(Kj  +...+  Ks)*)  «  c1(domjK|  +...+  domjK*s) 

iff 

reCj(Kj  +...+  K$)  »  reCjKj  +...+  reCjK$. 

PROOF.  8y  Theorems  4.6  and  2.6. 

The  next  theorem  parallels  the  result  obtained  by  Rockafellar  [38], 
Moreau  [32],  and  others  for  the  subdifferential  of  a  sum  of  convex  functions. 

THEOREM  4,9.  Assume  (*).  Then 

a(K1  +...+  Ks)(x,y)  ■  aK^x.y)  +  ...+  jKs(x,y) 
for  each  (x,y)  e  R01  *  Rn. 

PROOF.  Since  dom  (Kj  +...+  Ks)  ■  C  *  d  *  dom  K(0...ndom  K$,  (37.4) 
Implies  that  a(K^  +...+  K$)(x,y)  and  3Kj ( x ,y )  are  empty  (for  some  j) 
whenever  (x,y)  i  C  *  0.  So  suppose  that  (x,y)  c  C  *  D.  Then  Theorems  4.6 
and  2.3  imply  that 

D(K1  ♦...+  Ks)(x,y)  »  A*DK(A(x,y)). 

Tho  formula  follows  from  this  together  with  Theorem  4.4(d)  and  the  defini¬ 
tions,  after  observing  that  Af  and  A£  are  just  the  appropriate  addition 
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linear  transformations. 

The  next  theorem  identifies  certain  members  of  the  equivalence  class 
conjugate  to  [Kj  +...♦  K$]. 

THEOREM  4.10.  Assume  (#).  let  dom  K*  •  Cf  *  Df ,  and  define  func¬ 
tions  4  and  on  R111  x  Rn  by 

♦(z.w)  '  sup  inf  £]<*,(*<. w,) 

E  Zj"Z  Ew.«w 
z^eCf  1 

and 

<l>( z,w)  ■  inf  sup  SlKfU^wJ 

E Wj"W  E  Zj“Z  1  1  ' 

•M 

Then  [(K-j  +...+  K$)*J  contains  each  concave-convex  function  0  on  Rm  *  Rn 
satisfying  ♦  <  J  <,  *.  If  each  is  polyhedral,  then  [(Kj  K#)*] 
is.  polyhedral. 

PROOF.  By  Theorems  4.6  and  2.8,  [(K^  +...+  K$)*]  contains  each 
saddle-function  lying  between  two  certain  functions  Jj  and  Jg.  By  parts 
(e)  and  (c)  of  Theorem  4.4,  one  can  easily  show  that  Jj  ■  ♦  and  Jg  »  <j>. 

The  polyhedral  assertion  Is  Immediate  from  Corollary  4.6.1  and  the  fact 

that  [K*]  is  polyhedral  whenever  [K]  is. 

There  are  actually  many  representations  of  the  functions  t  and  ij<  In 
Theorem  4.10.  Suppose  and  are  any  sets  (not  necessarily  convex) 
such  that  ri  Cf  C  Zi  C  r"1  and  ri  Of  C  W.  C  Rn  (for  1  «  lf...,s). 

Then 

*(z,w)  s  sup  inf  SK^z^wJ 

EZj  =Z  EWj*W 
z^cCf  w^eW^ 

and 

i^(z,w)  »  inf  sup  Si»(z|.w<). 


55 


This  follows  from  the  observation  that,  If  A:  Rn  Rm  is  a  linear  trans¬ 
formation  and  f  is  a  proper  convex  function  on  Rn,  then  (Af)(y)  «* 

1  nf { f ( x) | Ax  ■  y,  x  c  C)  for  any  set  C  such  that  ri(dom  f)  c_  t  ~  Rn. 

The  fact  that  C(Kj  +. Ks)*l  contains  4  and  suggests  writing 
C(K1  ♦  K$)M  •  [K|]  D . . .0  [K*] 

and  calling  this  class  the  mlnimax  convolution  of  CKp,. . .  ,[KJ].  This  is 
the  saddle-function  analogue  of  the  operation  of  infimal  convolution  on  con¬ 
vex  functions.  The  identity  above  expresses  the  fact  that  the  operations 
of  addition  and  mlnimax  convolution  are  dual,  just  as  in  convex  function 
theory  the  tjrmula  (f|  +...+  f  )*  ■  ffrQ...Df£  expresses  the  duality  be¬ 
tween  the  operations  of  addition  and  infimal  convolution. 

The  next  theorem  gives  information  concerning  attainment  of  the  extrema 
appearing  in  the  definitions  of  t  and  *. 

THEOREM  4.11.  Let  $  an_d_  t>  be  def  ined  as  j_n  Theorem  4.10,  and  assume 
that  whenever  (z- ,W|)  c  ri(dom  Kp  for  i  -  the  following  condi¬ 

tions  are  satisfied: 

(a)  Iwj  =  0  and  X(rec  KpZj.OMw'p  <  0  =»  £(rec  K*(z. , • ) ) ( -w  1 )  <  0 

(b)  £.7^*0  and  £(rec  Kp*  twf))(z{)  >  0  £(rec  K*(*  ,wp)(-7p  >  0. 

Then  the  conclusions  of  Theorem  4.10  hold  and 

ri(dom  Kf  +...+  dom  Kp  C  dom  (K^  *...■►  Ks)*  d  dom  +.. .+  dom  K*. 
Moreover,  for  each  (z,w)  c  r1(dom  Kf  +. . .+  dom  Kp  there  exist  nonempty 
closed  convex  sets  Z  C  Rsm  and  W  C  Rsn  such  that  for  each 
( z^  , . . .  ,'z's )  c  Z  and  ( w  j , . . .  ,w$ )  c  W  the  following  statements  hold: 

(i)  £ ( z.j  )  »  (z.w)  and  (z^wp  c  dom  3K*  for  each  i; 

(ii)  *(z,w)  “  XKpz^.w,)  *  z ,w) ; 

(iil)  <  £Kpz.,w.)  <  SKpz-.wp  whenever  2(z.,w.)* 

(z.w)  and  (z^,wp  c  dom  K*  for  each  i. 
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PROOF.  liy  parts  (e)  and  (c)  of  Theorem  4.4  together  with  Lemma  4.3(f), 
A*  and  K*  «  (Kf,...,K*)  satisfy  the  hypotheses  of  Theorem  3.4.  Hie  asser¬ 
tions  are  immediate  from  this  and  Theorem  4.4(d). 

If  conditions  (a)  and  (b)  above  are  actually  satisfied  whenever 
(z-.w^)  e  dorn  K|  for  1  *  l,....s,  then  Theorem  3.5  implies  that 
dom  (K,  +...+  KJ*  *  dom  K?  +. ..+  dom  K*  and  that  a  and  ^  are  the  least 
and  greatest  members  of  C(Kij  <-...+  K$) *J. 

The  following  lenina  may  be  useful  in  applying  Theorem  4.11.  Notice  for 
example  that  its  conditions  are  satisfied  when  each  of  the  sets  dom  '(*  is 
bounded. 

LEMMA  4.12.  The_  following  condition  implies  that  condition  (a)  of 
Theorem  4.11  is  satisfied  for  each  choice  ot  e  dom^Kj,. . . ,zs  t  dom^X*: 

(c)  £  ■  0  and_  ws  c  0+  ci(dcm2K|)  tor 

each  i  imply  that,  w ■  -  0  f or  each  i . 

Similarly,  the  fol lowing  condition  implies  that  condition  (b)  of  Theorem  4_.1]_ 
is  satisfied  for  each  choice  of  e  dorn2K| , . . .  ,w^  e  dorn^X*: 

(d)  Zz.  =  0  and  z.  e  0f  cl{dom(Kt)  for 

each  i  imply  that  z.  3  0  for  each  i. 

PROOF.  Apply  Lemma  3.6  to  A*  and  K*  3  (K|,...,K*).  Condition  (c) 
(resp.  ( d) )  corresponds  to  condition  (c,)  (rasp,  (d,))  of  Lemma  3.S.,  and  con- 
dition  (a)  (resp.  (b) )  of  Theorem  4.11  corresponds  to  condition  ( n i ]  (resp. 
(b3))  of  Lemma  3.3. 

The  next  lemma  furnishes  alternate  characterizations  of  conditions  (c) 


and  (d)  of  Lemma  4.12 

LEMMA  4.13.  Le_t  P^». ..  ,PS  be  convex  cones  in  Rn  wh i ch  contain  the 
origin.  Then  the  following  condi lions  arc  egei valent : 

(i)  Zp,  “  0  and  p.  t  P.  for  each  i  imjHy  p.  --  0  for  each  i; 


(11)  (-PJn(convU P.)  •  (0)  for  each  J  • 

j  iW  i 

PROOF.  First,  observe  that  for  each  j,  (3.3)  implies 


convUP,  ■U(Za4P.|0<  x4  and  51*.  •  1). 
1f»J  1  Ipj  1  1  “  1  1 


From  this  It  follows  that  convLjP.  *  5lP4.  Thus,  condition  (11)  falls  Iff 

i»»j 1  ipj  1 


3j  and  9pj  c  Pj  such  that  0  p  -pj  c  XlPj. 

IpJ 


This  occurs  Iff 


3pj  e  P|,...,3ps  e  Ps  and 


3  t  such  that  0  p  -p4  *  IT.  p. , 
J  J  Ipj  1 


which  occurs  Iff  condition  (1)  falls. 

He  conclude  this  section  with  an  example  concerning  maximal  moriotcn* 
operators  arising  from  saddle-functions.  This  will  suggest  a  conjecture 
about  arbitrary  maximal  monotone  operators. 

By  (37.5.2),  each  closed  proper  concave-convex  function  K  on  Rm  *  Rn 
Induces  a  maximal  monotone  operator  T  (generally  multivalued)  from 
«r ,  r"  to  if*  x  R11  by  means  of  the  formula 

T(x,y)  •  {(-x*,y*)  j(x*,y*)  e  aK(x.y)}. 

By  (37.4.1),  T  depends  only  on  the  equivalence  class  containing  K.  It 
R(*)  denotes  the  range  of  an  operator  and  B  Is  the  linear  transformation 
which  sends  (x*,y*)  to  (-x#.y*)»  then  (37.5)  implies  that 

R(T)  -  8  dom  aK 

whenever  T  arises  from  K  as  above. 


EXAMPLE  4.14.  Assume  that  conditions  (c)  and  (d)  of  Lenina  4.12  are 
satisfied.  Then  by  Lemma  4.12  the  hypotheses  of  Theorem  4.11  are  met, 
and  these  in  turn  Imply  that  condition  (;>f)  Is  satisfied.  Let  &  be  the 
linear  transformation  defined  above,  let  be  the  maximal  monotone  opera¬ 
tor  Induced  by  K4  as  described  above,  and  similarly  (using  Theorem  4.5)  let 


T  be  the  maximal  monotone  operator  Induced  by  Zk^.  By  (37.4),  (6.3.1) 
and  (9,1)  it  follows  that  cl  R(T^)  *  B  cl  dotn  K^,  and  similarly  cl  R(T)  * 

B  cl  dw:(2(K^)*.  Theorem  4.11  and  (6.3.1)  imply  cl  dom(ZK.)*  *  cl  51  dom  K*. 
Combining  these  facts  with  (6.6-2)  yields  It  cl  fi(T^)  d  cl  R(T).  Since 
Theorem  4.7  Implies  2-Y.j  *=  T,  this  shows  that  %  is  a  maximal  monotone 
operator  satisfying 

2£c1  R(T{)  c:  cl  RtfE-Tj).  (!) 

Furthermore,  It  can  be  deduced  from  (1),  R(ZT^)  C  >~R(Tj)  and  (3.1)  th.tt 

S  0*  cl  K(T1)  C  0*  cl  R(S  T1 ) . 

It.  is  easy  to  show  that  conditions  (c)  and  (d)  of  Lemma  4.12  cnn  be  r'e formu¬ 
lated  equivalently  as  follows: 

51  Zj  ■  0  and  z1  *  0+  cl  R(T^) 
for  each  i  implies  *  0  for  each  i 
Now  (ft)  and  (9.1  1)  imply  £  cl  R(T^)  -  cl  I  R(T^) , 

R(JtlT^)  C  XR(T|)  C  5.  cl  R(T^)  holds  trivially.  Combining  these  facts 
with  (1;  yields 

2  cl  ?<lv )  »  cl  R(ZTj).  (?) 

Furthermore,  from  (2),  (ft)  and  (9.1.1)  it  follows  immediately  that 

£0>  cl  R(Ti)  *  0+  cl  R('21T<> . 

It  is  known  that  Is  a  maximal  monotone  operator  satisfying  (1) 

whenever  each  lb  is  the  subdi fferentfa!  of  a  closed  proper  convex  function 

on  Rn  and  the  condition 


} 


(ft) 


ri  D(Tj)ii...Pri  P(T$)  f  i  (t) 

is  satisfied,  where  0(T)  *  {z'f(z)  f  d>.  Moreover,  in  this  situation  (i) 
actually  holds  if  (ft)  is  satisfied. 

On  the  other  hand,  (1)  fails  in  general  for  maximal  monotone  operators 
satisfying  (t).  (For  example,  take  s  ■  2  and  consider  the  T.. * s  ’nduccd 
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by  the  saddle-functions  and  kg  defined  following  Corollary  4.6  3.) 

It  Is  not  known,  though,  whether  (2)  holds  for  arbitrary  maximal  monotone 
operators  satisfying  (+»).  But  the  fact  that  this  formula  does  hold  for 
those  operators  arising  from  saddle-functions  leads  one  to  conjecture  that 
It  holds  In  general.  This  Is  because  such  operators*  unlike  the  subdlfferen- 
tlals  of  convex  functions,  exhibit  most  of  the  pathology  of  arbitrary  maximal 
monotone  operators.  Indeed,  the  last  fact  Is  one  of  the  main  motivations  for 
studying  saddle-functions. 


S5:  The  °artfal  Conjugacy  Operation 

In  this  section  tne  results  of  *Z  are  used  to  develop  another  operation 
on  equivalence  classes  of  closed  proper  saddle-functions.  It  follows  from 
Theorems  5.1  and  5.2  that  this  operation  induces  a  symmetric  one-to-one 
correspondence  among  such  equivalence  classes.  In  f6  this  operation  is  used 
to  assign  a  well  defined  Ugrarvjian  to  each  dual  pair  of  generalized  saddle 
programs. 

Throughout  S5  let  K  be  a  closed  proper  concave-convex  function  on 
(Rp  x  Rm)  x  (Rq  x  Rn) ,  and  let  Wj  and  Wg  he  functions  cn  (P,p  *  Rn)  < 

(Rq  x  Rm)  defined  by 

Wj(u*,y,v*,x)  *  sup  inf  {<u*,u>  t  <v*,v>  -  K(u,x,v,y}> 

V  u 

and 

W9{u*,y,v*,x)  *  Inf  sup  (<u*,u>  +  <v*,v>  -  KtUvX.Vjyb. 

C  U  V 

THEOREM  5.1.  The  functions  end  belong  to  an  equivalenco  class 
[Vi]  cf  dosed  proper  concave-convex  functions.  Furthermore,  [W]  depends 
or)lL  on  [K],  and  01]  is  polyhedral  if  [K]  _1s. 

PROOF.  Define  a  linear  transformation  A  *  A,  *  and  a  function  H 

\  Cs 

OV 

* 

A|(vjU*,y)  a  (u*,y)» 

A0(u:v*,x)  *  (v+,x), 
uni 

H(v,u*,y,i»,v*vx)  -  <u,u*>  *  <v,v*>  -  K(u,x,v,y). 

Clearly  H  is  closed  prcpe*  concave-convex  on  {Rq  x  Rp  x  Rr')  x  (Rp  x  x  R111) . 
If  (v,y)  e  rifdoa^K)*  (34.3)  implies  that  the  function 

(u,»V<)  »  -K(u ,x,v,y) 

is  closed  proper  convex,  and  by  ( il. 5}  its  recessicn  function  crn  be  shown  to 

be 

(u,v*»x)  -v  -(rec  K{  •  ,*  ,v,y))(u,x) . 


ikamuSmmm 
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Also,  the  function 

(u,v*,x)  <u»u*>  +  <v,v*> 

Is  closed  proper  convex  and  coincides  with  its  recession  function.  Hence 
(9.3)  Implies  that 

(nec  H(v,u*,y,* ,• ,*))(u»v*,x)  ■  <u,u*>  +  <v,v*>  -(rec  K(*  ,•  ,v,y))(u,x) 
whenever  (v.y)  c  ri(domj>K).  Therefore  A^1  \ 0)  ri  (rec  conegH)  equals 
{(u,0,0) |<u,u*>  -  (rec  K(  ,*  v,y)){u,0)  <,  0,  V  u*  e  Rp,  V(v,y)  c  rltdonigK)}. 
How  by  (34.3)  and  (8.5),  (v.y)  r  .  (dom^K;  implies  that  rec  K(*,*,v,y)  Is 
never  +».  It  follows  that  .«2  tO>n(rec  conegH)  Is  the  nullspace  of 
Rp  *  Rq  *  r"1.  Similarly,  A^COlAOec  cone^H)  Is  the  nullspace  of 

x  Rp  x  Rn.  Therefore  by  Lemma  2.9,  range  A*r>r1(dom  H*)  ?•  0.  The  first 
two  assertions  of  the  theorem  row  follow  from  Theorem  2.8  and  the  fact  that 
whenever  Re  [K].  If  K  is  polyhedral,  then  Corollary  4.6.1 
Implies  H  is  polyhedral  and  hence  Theorem  2.8  implies  L'AH]  °  CW]  Is 
polyhedral. 

The  equivalence  class  [W]  .containing  Wj  and  W,,  Is  called  the 

partial  conjugate  of  CK]  in.  u  and  v,  and  the  operation  which  sends  [K] 

to  [W]  Is  called  partial  conjugate.  This  terminology  is  suggested  by  the 

fact  that  forming  [W]  Involves  only  parts  of  the  arguments  of  K,  whereas 

forming  the  (ordinary)  conjugate  CK*]  involves  all  of  the  arguments  of  K. 

THEOREM  5.2  The  partial  conjugate  of  [W]  in  u*  and  v*  .is  [K]. 

PROOF.  By  Theorem  5.1,  00  contains  the  function  Sf,  where 

R(u*,y,v*,x)  a  Inf  sup  {<^u,u*>  ♦  <v,v*>  -  K(u,x,v,y)).  Hence  the  partial 
u  v 

conjugate  of  £W]  In  u*  and  v*  contains  the  function  M  given  by 

M(u,x,v,y}  "  sup  inf  {<u,u*>  +  <v,v*>  -  P(u*,y,v*,x)} 
v*  u* 

*  sup  inf  sup  inf  {'U*,u  -  u>  +  <v*,v  -  v>  +  K(u,x,v,y) }. 

V*  U*  U  V 

By  the  same  technique  used  in  the  proof  of  Theorem  5.1  it  can  be  verified 
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that  range  8* A  n(dom  o*}  :f  where  P  -  8^  *  B 2  and  0  are  given  by 

i> . ( V **  »•: ,v»  t  k}  *  (U,X). 

B2(u*,v,vfy)  *  (v,y) , 
and 

J{v*,ii,u,x>u,\v,v,y)  *  <u*„u  -  u>  +  <v*.  v  -  v>  +  K(u»x,v,y) . 
therefore  Theorem  2.8  implies  that  [BJ]  is  well-defined,  dow  b>  •  .'6.1 ) 

and  Theorem  0.1(b)  it  follows  easily  r.hat  M  and  N  belong  to  P'J],  where 

N  is  given  by 

N(u,x,v,y)  »■  sup  inf  sup  inf  ;<u*,u  -  u>  +  <'*,v  -  v>  K(o,;uv,y)) .. 
u  v  v*  u* 

Thus,  to  complete  the  proof  it  suffices  to  show  tnat  N  t  [K]. 

Let  u„x,v,y  be  arbitrary  but  fixed.  For  such  j  define 

p(u)  =  inf  sup  inf  {<u*,u  ••  u>  +  <rv*,v  -  v>  +  K.(m,x » v »y) } . 
y  v*  u* 

Observe  that 

N(u,x.v,y)  *  sup{  p(C)|  5  el)},  (1) 

whsre  U  -  (  uj  (  j,x)  s  Indeed,  J f  (u,.s)  i  dorn^K  then 

K(u,x,’*,0  =  -«  so  that  p(ii)  *  Thus, 

N(j.x,v,y)  *  i.  *  K(u,x,v,y)  (2) 

whenever  U  *  N aw  assume  U  f  i.  For  each  ii  e  U.  K(u,x»'a»)  is  neve? 
-•»  and  hence 

p(u)  *  inf  sup  inf  {<u'-,u  -  u>  *  <v*,v  -  v>  f  M'J.x.v.y)}  > 
vtV(u)  v*  u* 

where  V(u)  -  (v|K(u,x,v,y)  <  >■*-}.  This  impl  ies  p(G)  *  ■»*'  whenever 
V(u)  *  i.  Hence  (1)  implies  N(u,x,v,y)  =  if  there  exist;,  a  u  «.  0 
such  that  Y(ii)  =  j5.  But  for  such  a  u,  K(C\x,v,y)  ~  Therefore 

N(u,x.v,y)  *  +«  *  K(u»x,v,yj  (3) 

whenever  there  exists  a  usU  such  that  V(v)  "  i.  Finally,  assure 
U  i  f>  and  V( u)  /  t>  for  every  u  c  U.  Then  for  each  £  ?.  V, 


p(u)  ■  .Inf.  (K(u,x,v,y)  +  sup  (<v*,v  -  v>  +  Inf  {<u*,u  -  u> } }> 
vcV( u)  “  v*  u* 

.  f-  'f  5|,“ 

l  _1nf  (K(u,x»v,y)  +  sup  {<v*,v  *  v>}}  if  u  3  u, 
veV(u)  ~  v* 


where 


p(u)  ■  inf  tK(u,x,v,y)|v  c  V(u),  v  ■  v) 
r  +»  If  V  ^  V(ll) 

^  K(u,x,v,y)  if  v  c  V(u). 

Hence  (1)  implies  that  In  this  case 

N(u,x,v,y)  ■  sup  (p(u)Ju  e  U,  u  *  u) 
/  —  i  f  u  i  U 


■<  +•  if  u  e  U  and  v  t  V(u) 

^  K(u,x,v,y)  If  u  e  U  and  v  e  V(u) 

3  K(u,x,v,y). 

Combining  this  with  (2)  and  (3)  yields  K  <  N  <.  ?  everywhere.  Hence  Theorem 
0.1(b)  implies  N  c  [K3. 


THEOREM  5 .  ? .  The  following  conditions  are  equivalent: 

(a)  (u*,x*tv*,y*)  e  3K(u,x,v,y) 

(b)  (u»-y*,v,-x*)  e  3W(u*,y,v*,x) 

PROOF.  By  (37.5)  condition  (b)  is  equivalent  to 

(u*,y>v*,x)  c  3W*(u,-y*,v,-x*). 

But  from  the  proof  of  Theorem  5.1  we  know  that  [W*]  =  [H*A*j  and 
range  A*r>ri(dom  H*)  t  d.  Hence  by  Theorem  2.3, 

3W*(u,-y*,v,-x*)  -  AaH*(A*(u,-y*,v,-x*)). 

It  follows  that  condition  (b)  is  equivalent  to  the  existence  of  points  >:c 
and  v°  such  that 

(v° ,u*,y,u°,v*,x)  e  3H*(o,u,-y*,o,v.-x*) . 
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A  n 

rtuc  by  (37/a)  and  (37.4)  this  containment  occurs  Iff  (v  »u*,y,u  ,v*,x) 
is  a  saddle-point  of 

H  -  <• »(o»u,-y*)>  *  <*.(o,v,  ~x*}> 

ind  H(vr 3u*,y,u° ,v*tx)  c  R.  Therefore  by  the  definition  of  H,  condition 
(b)  vs  equivalent  tc  the  existence  of  points  u°  and  v°  such  that 
K(u3>xtv°»y)  c  R  and 

K(u,x,v°,y)  -  <u  -  u,u*>  -  <v°  -  v,v*>  -  vx  -  x,x*> 

<_  K(u°,x,v°,y)  -  -  u»u*>  -  <v°  -  v,v*> 

<  K(u0,x,Vj,y}  -  <u°  -  u5u*>  -  <v  -  v,v*>  -  <y  -  y,y*> 

for  all  (v,u*,y)  and  (u,v*9x).  Now  pick  ar.y  ( v * sy ' )  c  dorii^K,  Choosing 
v  «  v'  and  y  »y!  in  the  above  condition  implies 

K(u°  ,x,v'  r,y ' )  >  a  v  cu°  -  utu*>  for  all  u*9 
where  «  is  a  certain  real  constant.  Thus  If  u°  were  different  from  u, 
we  would  have  K(u°  ,x,v‘  ,y’ )  *  +»,  contradicting  (v',y‘)  e  don^K.  Hence 
in  the  above  condition  we  must  have  u°  *  u,  and  similarly  v°  3  v.  There¬ 
for*  condition  (h)  is  equivalent  to  (K(u»x»v,y)  c  K  and) 

K(u./,v,y)  -  <u  -  utg*>  -  <x  -  x,x*>  <  K(u,x,v„y),  \/(u,x) 
and 

K(u,r,v,y)  £  K(u,x,v,y)  -  <v  -  v,v*>  -  <y  -  y,y*>.  V(v,y). 

Hut  these  conditions  are  clearly  equivalent  to  (a). 


lilLMMtWIUiaMMI 
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16:  Generalized  Saddle  Programs 

In  this  section  the  results  of  f§2,  3  and  5  are  applied  to  the  problem 
of  associating  with  a  given  minimax  problem  a  dual  minimax  problem  and 
developing  a  perturbatlonal  duality  theory  for  such  pairs  of  problems. 

Ignoring  technical  details,  we  can  outline  the  general  approach  as 
follows.  Suppose  we  are  given  a  minimax  problem  In  the  form  of  an  equlva- 
lence  class  CK<)3  of  saddle-functions  on  R*11  *  Rn.  This  minimax  problem 
Is  first  extended  to  a  saddle  program  In  the  form  of  another  equivalence 
class  [K]  of  saddle-functions  on  (R^  *  r”1)  *  (R^  *  Rn).  The  extension 
Is  such  that  [Ko3  Is  suitably  "embedded"  In  [K3»  l.e.,  the  saddle-func¬ 
tions  (x,y)  ♦  5f(o,x,o,y)  for  1 t  [K]  are  all  required  to  belong  to  [K0]. 
By  a  modification  of  the  conjugacy  correspondence,  an  equivalence  class  [Lj 
of  saddle-functions  on  (r"*  *  Rp)  *  {Rn  x  Rq)  Is  then  obtained  from  [K]. 

The  saddle  program  given  by  [1]  Is  called  the  dual  of  the  program  given 
by  [K].  Under  a  mild  hypothesis  on  [K],  the  saddle-functions 
(z,w)  •*  r(o,z,o,w)  for  T  c  [L]  all  belong  to  a  single  equivalence  class 
[L01.  In  this  event  the  minimax  problem  given  by  C L-o D  is  the  dual  of  the 
minimax  problem  given  by  [Kg],  In  this  sense  LKo3  may  have  many  such 
duals,  since  CL3  and  hence  fl03  depends  not  only  on  CK0 1  but  also  on 
the  particular  "perturbations"  introduced  via  [KJ. 

We  proceed  now  with  the  formal  development.  A  generalized  saddle  pro¬ 
gram  S(K)  on  Rf”  x  Rn  with  perturbations  In  R*1*  ^  Is  a  closed  proper 
saddle-function  K.  on  (R^  *  fi”1)  x  (R**  x  Rn).  fach  saddle-function 
K(o,*  ,o,«)  on  r”1  x  Rn  for  E  In  [K]  is  called  an  objective  function 
of  S(K).  The  particular  functions  K(o,*,o,»)  and  K(o,*,o,*)  are  called 
the  lower  and  upper  objective  functions,  respectively.  (Recall  the  conven¬ 
tion  established  In  Theorem  0.1:  for  a  closed  saddle-function  K,  its 
convex-closure  Is  denoted  by  K  and  Its  concave-closure  by  hf. )  A  pair 


(x,y)  Is  a  feasible  solution  of  S(K)  Iff  It  Is  in  the  domain  of  every  ob¬ 
jective  function  of  5(K).  It  Is  not  hard  to  show  that  this  is  equivalent 
to  the  condition  that  (o,x,o,y)  e  dom  K.  The  optimal  value  in  S ( K)  exists 
{and  equals  m)  Iff  the  saddle-values  of  all  the  objective  functions  of  S(K) 
exist  finitely  and  are  equal  (to  a).  A  pair  (x,y)  is  an  optimal  solution 
of  5(K)  iff  (x,y)  Is  a  saddle-point  of  every  objective  function  of  S( K } 
and  K{o,x,o,y)  *  f(o,x,o,y)  t  R.  It  is  not  hard  to  show  that  if  (x,.y)  is 
ar.  optimal  solution,  then  it  is  a  feasible  solution  and  the  optimal  value 
exists  and  equals  K(o,x,o,y)  for  any  ft  in  [K]. 

The  program  S( K)  is  consistent  (respectively  strongly  consistent)  Iff 
there  exist  points  x  and  y  such  that  (o,x,o,y)  e  dom  K  (respectively 
(o,x,o,y)  r.  ri(dom  K) ) ..  Thus,  S(K)  is  consistent  iff  It  has  a  feasible 
solution.  Also,  S(K)  is  consistent  whenever  the  optimal  value  in  $(K) 
exists. 

We  say  that  S(K)  has  a  well  -defined  primal  problem  Iff  all  the  objec¬ 
tive  functions  belong  to  the  same  equivalence  class,  which  we  denote  by  f K p 'J - 
In  this  event  the  definitions  of  feasible  solutions,  optimal  value  and  optimal 
solutions  of  S(K)  can  be  simplified,  since  equivalent  saddle-functions  have 
the  same  domain,  saddle-value  and  saddle-points.  By  Theorem  6.?  below,  if 
S(K)  is  strongly  consistent  then  it  has  a  well-defined  primal  problem  which 
Is  in  fact  given  by  a  closed  proper  equivalence  class.  More  genera'ly,  for 
any  (u,v)  we  say  that  the  perturbation  (u,v)  jn.  S(K)  is  well-defined 
iff  the  saddle-functions  ft(u,  ,v,*)  on  ff1  *  Rn,  for  ft  in  [K],  all  belong 
to  a  single  equivalence  class,  which  we  denote  by  [K  ].  Thus,  5(K)  has 

U  |  V 

a  well-defined  primal  problem  iff  the  perturbation  (o»o)  in  $(K)  is  well- 
defined  (in  which  case  IK.  3  is  denoted  simply  by  [Koj). 

0  k  w 

Suppose  S( K)  is  a  generalised  saddle  program  on  Rm  x  Kn  with 
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perturbations  In  fP  x  if,  tnd  let  [1]  be  the  equivalence  class  of  closed 
proper  saddle-functions  obtained  from  [K]  via  the  relation 

L(s,z,t,w)  ■  -K*(-z,s,-w,t). 

The  generalized  saddle  program  $(L)  on  R*5  x  if  with  perturbations  in 
if  x  Rn  Is  the  dual  of  ${K).  It  Is  easy  to  show  that  the  dual  of  S(L) 

Is  S(K).  The  program  S(K)  has  a  well-defined  dual  problem  Iff  the  dual 
program  S(L)  has  a  well-defined  primal  problem  CL0L  and  In  this  event 
the  dual  problem  of  S(K)  Is  the  minimax  problem  given  by  CL03-  Example  6.3 
shows  that  a  generalized  saddle  program  can  even  be  strongly  consistent  with¬ 
out  having  a  well-defined  dual  problem.  However,  Lemma  6.4  furnishes  condi¬ 
tions  on  S(K)  which  ensure  that  the  dual  problem  Is  well-defined. 

For  the  remainder  of  16  let  S(K)  and  S(L)  be  dual  programs,  where 

for  definiteness  K  Is  assumed  to  be  concave-convex  on  (Rp  *  if)  *  (Rq  *  Rn). 

Thus,  L  Is  convex-concave  on  (if  *  rP)  *  (R0  *  Rq).  Also,  let  concave- 

convex  functions  Pj  and  Pg  *>•  defined  on  Rp  *  Rq  by 

P,(u,v)  ■  sup  Inf  K(u,x,v,y) 

1  x  y 


and 


P2(u.v)  ■ 

and  let  convex-concave  functions 

Q,(s.t)  » 

and 


Inf  sup  K(u,x,v,y), 

y  * 

Q1  and  Qg  be  defined  on 

sup  Inf  L,(s,z,t,w) 
w  z 


Rm  x  Rn 


by 


q2(s,t)  ■  Inf  sup  U(s,z,t,w). 
c  z  w 

Finally,  let  linear  transformations  A^:  Rp  x  Rm  Rp,  Ag:  Rq  *  Rn  Rq, 
if  x  Rp  +  if  and  Bg:  Rn  x  Rq  ♦  if  be  defined  by 

Aj(u,x)  ■  u,  8j(s,z)  ■  s, 

Ag(v,y)  ■  v,  Bg(t,w)  .■  t, 

and  put  A  ■  A1  x  Ag  and  B  ■  B^  x  Bg.  Observe  that  A*  «  Af  *  Ag  and 
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B*  «  Bf  *  Bg,  where 

Af<z)  *  (2,0),  Bf(x)  »  (x,o) 

A|(w)  ■  (w,o) t  Bg(y)  «  (y,o). 

The  saddle-functions  and  P2  are  called  the  lower  and  upper  per¬ 
turbation  functions  of  S(K),  respectively.  A  pair  (z,w)  Isa  Kuhrv-Tucker 
vector  for  ${K)  Iff 

P|(u,o)  *  PgCo.o)  MtK 
and 

<u»z>  +  P„(u  o)  <<X  <  P.(o.v)  ♦  <v*w> 

Ik  I 

for  each  (u,v).  Observe  that  Pj(OiO)  *  P2(o,o)  •  a  e  R  Iff  the  optimal 
value  In  S(K)  exists  and  equals  a.  It  Is  not  hard  to  show  using  (37.4.1) 
that  if  P,  and  P2  belong  tc  a  proper  equivalence  class  [P],  then  (z,w) 

Is  a  KuhrHfucker  vector  for  S(K)  iff  -(z,w)  s  3P(o,o).  Kuhn-Tucker  vectors 
for  S(L)  are  defined  similarly  by  using  the  lower, and  upper  perturbation 
functions  of  S(L),  1.9.,  Qj  and  Q2.  These  Kuhn-Tucker  vectors  can  be  In¬ 
terpreted  as  generalized  "equilibrium  price  vectors"  in  much  the  saint*  way  as 
In  M4,  pp.  295-296.1. 

The  following  example  shows  that  the  foregoing  framework  of  dual  pairs  of 
generalized  saddle-programs  includes  as  a  special  case  Rock a  ft  liar's  dual 
pairs  of  generalized  convex  programs. 

EXAMPLE  6.1.  Let.  F:  •+•  Rn  be  a  closed  proper  convex  bi function »  and 

let  (P)  and  (P*)  denote  the  generalized  convex  program  and  Its  dual  which 
correspond  to  F  and  Its  adjoint  bi  function  F*:  R1'  •+  R^.  Define 
K(u,x,v,y)  o  (Fv)(y)  for  every  (u,x)  s  Rp  x  if  and  (v,y)  c  R4*  *  Rn 
(here  p  and  m  can  be  any  positive  integers).  Then  K  is  a  closed  proper 
concave-convex  function.  It  can  be  verified  as  an  instructive  exerciser  that 
the  concepts  defined  above  for  the  program  S(K)  and  Its  dual  S(l.)  exactly 


"mirror11  the  like-named  concepts  from  Rockefeller  [44]  for  (P)  and  (P*). 

For  example.  S(K)  (resp.  S( L) )  Is  consistent  or  strongly  consistent  ac¬ 
cording  as  (P)  (resp.  (P*))  Is  consistent  or  strongly  consistent;  and  so 
on.  Furthermore.  It  can  be  seen  that  the  Lagranglan  saddle-function  M 
associated  with  S(K)  and  S(L).  which  will  be  Introduced  following  Theorem 
6.8,  exactly  mirrors  the  Lagranglan  associated  with  (P)  and  (P*).  The 
fact  that  all  the  various  concepts  associated  with  (P)  and  (P*)  are  re¬ 
flected  In  this  program  S(K)  and  Its  dual  furnishes  a  general  means  of  con¬ 
verting  examples  from  convex  programming  into  examples  In  saddle  programing 
which  exhibit  the  corresponding  pathological  behavior. 

THEOREM  6.2.  Assume  (u.v)  c  A  r1(dom  K) ,  l.e.  assume  there  exist  points 
x  y  such  that  (u.x.v.y)  t  ri(dom  K).  Then  the  perturbation  (u,v)  jm 

S(K)  is  well-defined.  In  fact,  the  equivalence  class  tky  y]  Is  closed  and 
proper  with  least  and  greatest  members  K(u,*,v,*)  and  K(u,*,v,*)  respec- 
tl vely,  and 

r1(dom  K  )  ■  {(x.y) ((u,x,v,y)  c  r1(dom  K) } 

U  »  V 

where  "rl"  can  be  deleted  or  replaced  bj£  "cl"  throughout  the  Identity. 

PROOF.  Define  linear  transformations  Tj :  Rm -*•  Rp  x  r"1  and 
Tg:  Rn  -*■  Rq  x  Rn  by  Tj(x)  ■  (o.x)  and  T2(y)  a  (o.y),  and  put  T  =  T|  x  T^. 
Define  a  function  H  by 

H(u',x',v',y')  a  K( u *  ♦  u.x'.v'  +  v.y1). 

Clearly,  H  Is  closed  proper  concave-convex  and  dora  H  *  dom  K  -  {(u,o,v,o)}. 
Thus  the  hypothesis  on  (u.v)  is  equivalent  to  range  T  n  r1(dom  H)  /  f>,  and 
hence  Theorem  2.2  Implies  various  facts  about  the  equivalence  class  CHT]. 
Since  HT  »  K(u,«,v,*)»  these  facts  convert  easily  Into  the  assertions  about 
[K  UJ.  The  formulas  for  ri(dom  K  )  and  cl(dom  K  )  follow  by  (6.7). 

COROLLARY  6.2.1.  Assume  S(K)  Is  strongly  consistent  and  that 
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rec  cOMjKo  j_s  a  subspace  for  j  *  I  and  2 .  Then  there  exists  an  optimal 
solution  of  S(K) . 

PROOF.  By  Theorem  6.2,  S(K)  has  a  well-defined  primal  problem  and 
[Kq]  is  closed  and  proper.  Bv  Lemma  0.4.  (o,o)  e  ri(dom(K0)*) .  Hence 
(37.5.3)  Implies  K0  has  a  saddle-point. 

Before  proceeding  any  further,  it  might  be  well  to  illustrate  some  of 
the  pathology  which  Is  possible  In  a  dual  pair  of  generalized  saddle  programs 
The  next  example  exhibits  a  program  S(K)  having  the  following  properties: 

(1)  every  perturbation  in  S(K)  is  well-defined  (so  a  fortiori  S(K)  has  a 
well-defined  primal  problem);  (2)  the  lower  and  upper  perturbation  functions 
of  S(K)  fail  to  be  equivalent;  (3)  the  dual  program  is  consistent;  and 
(4)  S(K)  fails  to  have  a  well-defined  dual  problem. 

EXAMPLE  6.3.  Suppose  J  is  a  closed  proper  concave-convex  function  on 
r”1  x  Rn.  Put  p  ■  m  and  q  ■  n  and  define  K(u,x,v,y)  *  J(x  -  u,y  -  v) . 

Let  Tj  and  Tg  be  linear  transformations  given  by  f^(u,x)  «  x  ••  u  and 
Tg(v,y)  "  y  -  v,  and  put  T  -  T.(  «  T^.  Since  range  Tmi(dom  J)  /  d 
trivially,  Theorem  2.2  implies  that  K  *  J7  Is  closed  and  proper  with 

ri(dom  K)  »  T"Ari(dom  J).  By  Theorem  6.2  it  follows  that  for  each 

* 

(u,v)  c  Rp  x  Rq  the  perturbation  (u.v)  in  the  program  S(K)  is  well-defined. 
It  is  easy  to  compute  that  Pj(u,v)  -  sup  inf  J  »  -J*(o,o)  and  P2(u,v)  B 
inf  sup  J  3  -J*(o,o).  Hence  P.,^  Tg  iff  J*(o,o)  *  J*(o,o).  Now  suppose 
J  is  such  that  dom  J*  is  bounded.  Then  Lemma  '3.6  and  Theorem  3.5  imply 
that  [K*]  «  dom  K*  «*  v*dom  J*,  *nd  (since  T|(s)  «*  (-s,s), 

T|(t)  3  ( -t , t) )  the  least  and  greatest  members  of  [K*]  are 

K*(z,s ,w,t)  --1  «;up  *nf  .l_*(x,t) 

t S  j -5“2 J  tt | -t*=W) 

and 

K*(z,s,w,t)  -  inf  sup  J*(s,t). 

1 1 J  t-W}  { S I  • S ^ Z } 


jifaiatia Omm**** — 


'i  ■  ./  r  .. .  m  , 
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Since  L(s»z»t»w)  *  -F*(-z,s,-w,t)  and 

U(s,z,t,w) 

*  -K*(-Z,s 

Implies  that 

If  s  ■  z 

and  t  ■  w 

^(s.z.t.w)  •  ^  *• 

if  s  f  z 

and  t  ■  w 

If 

t  f  w 

and 

if  s«z 

and  t  *  w 

t(s,z,t,w)  ■  f  — 

If  s  «  z 

and  t  f  w 

*  +» 

If  s  ft 

From  these  formulas  It  follows  that,  for  each  (s,t)  c  dom  J*,  the  perturba¬ 
tion  (s,t)  In  $(L)  Is  well-defined  iff  J*(s,t)  *  T*(s,t) .  In  view  of 
all  these  facts,  In  order  to  obtain  properties  (1)  through  (4)  we  need  only 
specify  a  J  such  that  dom  J*  1$  bounded,  (o,o)  c  dom  J*,  and  J*(o,o)  f 
3*(o,o).  It  suffices  to  take  [Jj  to  be  the  conjugate  of  the  equivalence 
class  used  In  Examples  2.10  and  2.11. 

While  wt  are  concerned  mainly  with  applying  the  results  of  y§2,  3  and  5 
to  dual  pairs  of  generalized  saddle  programs,  we  note  here  some  of  the  results 
that  follow  from  |1.  From  Lemma  1.7  It  can  be  deduced  that  In  general 
L(o»* ,o,»)  <  (-Pj)*  <  Glty*  £ r(o,-,o,*) 

and  dually 

K(o,*,o,*)  <  (-Qj)*  i  £  Ro,* ,o,'). 

If  S(L)  has  a  well-defined  primal  problem  given  by  [L0J  and  [LoD  is 
closed,  then  it  can  be  deduced  from  Theorem  1.1  that  each  concave-convex 
function  P  satisfying  Pj  £  P  £  is  si,nP^e  an<i  satisfies 

cljcy  ■  -(Q*.  cl,d2P  -  -(LJ*, 
dom(cl2cl]P)  *  dom(L0)*  »  dom(cl ^cl gP) , 


where 
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dom(Lo)*  C  cU(u,v)!(u,x,v,y)  c  dotn  K  for  some  x,y). 

If  in  addition  CL<>3  Is  proper  (l.e.  S(l.)  Is  consistent),  then  P  Is  also 
proper  and  has  the  same  kernel  as  -(L0)*. 

By  Theorem  6.2,  S(K)  has  a  well-defined  dual  problem  [L0]  whenever 
S(l)  Is  strongly  consistent.  The  next  lemma  dualizes  this  useful  condition. 

LEMMA  6.4.  S{L)  jjs  strongly  consistent  Iff 

(rec^K)(o,x)  i  U  implies  (rec^K)(o,-x)  0 

and 

(rec2K)(o,y)  <  0  implies  (ree2K)(o,-y)  <  0. 

PROOF.  Observe  that  L(o,z,o,w)  *  -R*A*(~z,-w).  Hence  S(L.)  is  strong¬ 
ly  consistent  Iff  range  A*r»ri(dom  K*)  f  j t.  Now  apply  the  eou\' valence  be¬ 
tween  (a)  and  (c)  of  Lennia  2.9. 

THEOREM  6.5.  Assume  S(Ll  is  strongly  consistent.  Then  P-  and  P., 
belong  to  the  closed  proper  equivalence  class  CP]  *  [~(LC)*J  and 
dom  P  C  A  dom  K, 

PROOF.  By  Theorem  C.2,  L(o,*,o,«)  is  the  least  member  of  [L0]» 
which  is  closed  and  proper.  Hence  -L.(o,-z,o,-w)  *  -Ljj.1  -z , -w}  *  (-(U)»)*?z,w) . 
But  as  noted  in  the  proof  of  Lemma  6.4,  S(L)  is  strongly  consistent  Iff 
range  A *n  ri(dom  K*)  f  d*  and  K*A*(z,w)  a  -JL(o,-z,o,-w).  Hence  Theorem  2.8 
implies  that  the  equivalence  class  fAKJ  is  well-defined  and  equals  C -( L0 ) *3 * 
and  dom  AK  c:  A  dcm  K.  How  observe  that 

P,('u,v)  *  sup  inf  K,  P,(u,v)  *  inf  sup  K. 

1  A^tu)  A‘lfv)  c  A^  {v>  A^(u) 

Thus  and  P2  belong  to  [AKj.  Taking  [P]  3  [AK],  the  theorem 
follows. 

COROLLARY  6.5.1.  Assume  S(l)  is  strongly  consistent.  Then  the  follow¬ 
ing  conditions  on  (z,w)  e  *  R°  are  equivalent: 

(i)  (z.w)  is  an  optimal  sciutlon  of  S(L); 
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(ii)  (**#)  is  a  KuhrHTuckcr  vector  for  S( K> ; 

(111)  -U»w)  c  3P(o,o); 

(1v)  (-z,o,-w,o)  e  aK(o,x,o,y)  for  some  (x,y)  e  (f*  *  Rn. 

PROOF.  By  (37.5)  and  Theorem  6.5,  (i)  is  equivalent  to 
(z,w)  c  3(-P)(o,o),  which  Is  equivalent  to  (lit).  Since  Pj  and  belong 
to  CP]i  (37.4.1)  Implies  that  ?P(o,o)  a  8^(0, o)  *  8^(0, o)  and 
Pj(o,o)  ■  P?(o,o)  *  °*  ^s0*  (37.4)  Implies  dom  aP  c  dom  P,  so  that  a  is 

finite.  From  these  facts  it  follows  that  (Hi)  is  equivalent  to  (ii).  Fin¬ 
ally,  observe  that  (37  5)  implies  (til)  is  equivalent  to  (o,o)  r.  aP*(-z,-w). 
Since  [P*J  ■  [K*A*J  by  the  proof  of  Theorem  6.5,  Theorem  2.3  implies  that 
aP*(-z,-w)  -  AaK*(A*(-z,-h>).  Hence  (o.o)  c  aP*(-z,-w)  is  equivalent  to 
the  existence  of  (u,x,v,y)  e  8K*(-z.o,-w,o)  such  that  Aj(u,x)  -  0  and 
A?(v,y)  -  C.  By  the  definitions  of  A-  and  A^  and  (37.5),  this  last  con¬ 
dition  is  equivalent  to  (iv). 

COROLLARY  6.5.c.  Assume  S(L)  is  strongly  consistent,  and  let  L'P]  be 

the  equivalence  class  containing  P^  and  P?.  Then 

sup  itif  lu  -  P(o,o)  <_  P( o , v)  =  inf  sup  L0. 

Furthermore,  for  any  P  e.  L P] , 

sup  inf  t0  *  lim  inf  P(o,v) 
v  *  O 

whenever  -»  <  sup  inf  l0  or  f(o,o)  <  +«,  and 

11m  sup  P(u,o)  a  Inf  sup  t0 
u  -*  o 

whenever  —>  <  P(o,o)  or  Inf  sup  t0  «.  +». 

PROOF.  Clearly  sup  Inf  L0  ••  '(ro~)*(o,o)  which  equals  P(o,o)  by 
Theorem  6.5.  Similarly,  Inf  sup  L<,  =  -{U.)*(o,o)  s  P(o,o).  Now  let  P  e  [P] 
be  given.  By  Theorem  0.1,  P(o.o)  =  (c1?P)(o,o),  which  by  definition  equals 
(cl  ?{o,'))(o).  Now  in  general,  for  a  convex  function  f  one  has  (c)  t)(x) 
lim  Inf  f(y)  except  when  (cl  f)(x)  s  -«■  and  f(x)  s  +**.  Applying  this  fart 

y  *  x 
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to  the  case  at  hand  yields  that  sup  inf  Lo  *  (cl  P(o, •))(<>)  •-  lim  irf  F(o,v) 

v  +  o 

except  when  sup  inf  l0  ■  -»  and  P(o,o)  »  +».  The  remaining  assertion 
follows  similarly. 

COROLLARY  6.5.3.  Assume  S(l)  Is  strongly  consistent.  If  the  optimal 
value  In  S(L)  exists  and  equals  a.  then  the  optimal  value  in  S(K)  exists 
and  equals  a. 

PROOF.  Since  the  saddle- functions  U(o,*,o»*)  for  L  in  [LJ  are  all 
equivalent  to  L0»  sup  1  r.f  Lj  *  0^(o,o)  and  inf  sup  Lo  *  Q9(o,o).  Also,  the 
optimal  value  in  S{1)  exists  and  equals  a  Iff  U^(o,o)  *  Qp(o,o)  *  o  r  R. 
Since  P,  <  P^  <_  P?  i  The  assertion  now  follows  from  Corollary  6.5  2. 

COROLLARY  6. 5. a.  Assume  SfK)  :s  strongly  consistent.  In  order  that 

ri{!>  den  L)  er  dom  <}C  B  dorr  l, 
it  is  necessary  and  sufficient  that. 

(reCjK)(o,>)  =  >ec,Kc}(x) ,  Vx  e  Rm 
and 

(reCjf.) {o,y)  *  ( rec,,K0)(y)  ,  Vy  e  Rn. 

PROOF.  Dualizing  the  proof  of  Theorem  6,5  shows  that 

range  Q*C\  ri(dom  L*)  f  u  and  dewn  Kfl  s  dom  C  B  dc»n  L.  Hence  oy  Theorem 

2.6  and  (6.3.1),  rl (B  dom  L)  C  dom  K f  C  B  dom  L  occurs  iff  reCj(L*B*)  = 

vrec.L*)Bf  for  j  -  1  ar.d  2.  But  the  i denticles  *(rec,K)(o,-x)  v 
j  .1  * 

(reCjLlV)(Sfx)  and  -(rec^K0!(-x)  =  (reCjL*P*)(x)  can  be  verified,  along 
with  simi*ar  identities  for  j  *  2.  The  corollary  then  follows. 

example  6.14  demonstrates  that  the  domain  inclusion  in  Theorem  6.5  can¬ 
not  be  strengthened  to  equality  without  a  stronger  hypothesis.  For  any  pro¬ 
duct  set  C  x  D  contained  in  dom  K,  let  Hyp(C  *  D)  denote  the  following 
hypothesis: 

V(u,x)  c  C,  (rec  K(u,x,‘ , - ) ) ( o ,y }  _<_  0  (rec  K(u,x,*  ,-))(c,-y)  <.  C; 


•*VW  IMiWl 
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and 

V(v,y)  e  D,  (rec  ?(*,*. v,y))(o,x)  >  0  (rec  K(- .*  ,v,y))(o,-x)  >  0. 
If  Hyp(C  *  0)  for  some  C*  0  3  ri(dom  K),  then  Lemma  6.4  implies  that 
S(L)  Is  strongly  consistent. 

LEMMA  6.6.  The  following  conditions  are  equivalent,  and  they  imply 


Hyp(dom  K) : 

C  (o,x)  e  o+(cl(dom,K))  x  »  o 

(a)  +  1 

1  (o,y)  c  o  (c1(dom2K))  y  »  o 

/■There  exist  points  u  and  v  such  that  the  sets 

(b)  <  ({u}  x  rf”)  D  rl(dom^K)  and  ({v}  x  Rn)On'(dom2K) 

^  are  nonempty  and  bounded. 

PROOF.  By  Lemma  3.6. 

THEOREM  6.7.  Assume  Hyp(r1(dom  K)).  Then  the  conclusions  of  Theorem 
6.S  hold,  and  in  addition  A  r1(dom  K)  C  dom  ?  C  A  dom  K. 

For  each  (u.v)  c  A  ri(dom  K),  the  perturbation  (u,v)  S(K) 
is  well-defined.  [Ky  y]  is.  closed  and  proper,  and  the  set  of  saddle- 
points  of  [Kuy]  is  nonempty,  each  such  saddle-point  (x.y)  satisfies 
(u.x.v.y)  e  dom  3K  and  P(u,v)  ■  K(u,x,v,y)  for  every  Pc  [P'J  and_  Ke  [KJ. 

PROOF,  By  Lemma  6.4,  Hyp(r1(dom  K))  implies  S(L)  is  strongly  consis¬ 
tent.  Hence  the  conclusions  of  Theorem  6.5  hold,  and  in  particular 
domjPj  x  domgPg  *  dom  P.  Thus  it  follows  from  Hyp(ri(dom  K))  and  Lemma  3.3 
that  A  ri(dom  K)  C  dom  P.  If  (u,v)  t  A  ri(dom  K),  then  Theorem  6.2  implies 
the  perturbation  (u.v)  Is  well-defined  and  that  [K  j  is  closed  and  proper. 

U  >  V 

The  assertions  concerning  the  saddle-points  of  [K  J  are  immediate  from 

U  j  V 

Hyp(ri(dom  K))  and  Theorem  3.4. 

If  actually  Hyp(dom  K),  then  Lenina  3.3  and  Theorem  3.5  imply  that 
dom  P  3  A  dom  K  and  that  P 


1 


and  ?2  are  the  least  and  greatest  members 


of  [P],  respectively. 
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COROLLARY  6.7.1.  Assutne  S(K)  Is  strongly  consistent  a.id  sitter 
dom  X<j  Is  bounded  or  H,yp( ri ( 000?  K) ) .  Then  there  exist  optima*  solutions  of 
both  S(K)  and  S ( l)  ,  and  the  optimal  values  in  the  two  programs  are  eguaj_, 
Moreover,  the  optimal  solutions  of  one  program  are  precisely  the  Kuh.T-Tucfcer 
vectors  for  j.he  other. 

PROOF,  3y  Theorem  6  2  and  lemma  <5.6,  if  S(K).  ir,  strongly  consistent 
and  dom  is  bounded,  then  Hyp  (dom  K),  and  hence  Hyp(H(dora  K)).  dy 
Theorem.  6.7  and  (6.3.1J,  S(K)  is  strongly  consistent  iff  (c,o)  r-  ri (dom  P)  * 
A  ri(dom  K).  Hence  Theorem  6.7  implies  there  exists  e  sac'ole -point  of  L'Koj. 
i.e.  an  optimal  solution  of  $(K>.  Also,  ri(dom  P)  C  dcm  9P  ty  so 

that  aP(o.c)  /  p.  ty  Corollary  6.5.1  this  implies  there  exists  v\  optima) 
solution  of  S ( L ) .  Since  both  programs  are  strongly  consistent.  Corollary 
6.5.1  implies  that  the  optimal  solutions  of  one  are  the  Kuhn-1  ticker  vectors 
for  the  other.  The  two  optimal  values  are  equal  by  Corollary  6.5.3. 

As  a  criterion  guaranteeing  the  existence  of  an  optimal  solution  of  S(Kl 
the  hypothesis  0Y*  Corollary  6.7  1  is  stronger  than  necessary.  This  is  clear 
from  the  next  lemma  and  Corollary  6.2. 1.  In  fact,  as  will  become  cl.sar  later 
in  the  section,  tue  hypothesis  of  Core  Mary  5.2.1  suffices  fer  the  first  ease.’ 
tion  of  Corollary  6.7.1. 

LEMMA  6.8.  Assume  S(?)  Is  strongly  consistent.  Then  Hyp(ri(dcr  K)) 
implies  that,  rec  tone  j  .To  is,  a  subspace  fox  j  -  1  and  2 

PROOF.  Let  T  be  as  in  tne  proo-’  of  Theorem  5.2.  Then  strong  consis¬ 
tency  of  S(iC;  is  equivalent  to  range  T  ry  n(dom  K)  H  $,  end  hence  r<d  ;  - 
CKT]  by  Theorem  2.2.  Therefore  the  formulas  in  Lemma  2.5  cin  be  applied  to 
show  toot 

rec  cone,K0  *  (xj(o,x)  e  rec  cone  ¥.{*  ,«,o,y)  whenever  (c.y-  5:  ri(r!o..v:</ } 

and 
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rec  conegKo  -  ^yl(o.y)  c  rec  cone  K(o,x,« »•)  whenever  (o,x)  e  ri(dom^K)}. 
For  these  two  sets  to  be  subspaces  It  suffices  to  show  they  are  closed  under 
scaler  multiplication  by  -1,  and  this  follows  immediately  from  strong  consis¬ 
tency  and  Hyp(r1(dotn  K)). 

Let  M  be  defined  on  (tf"  *  Rq)  *  (Rn  x  rP)  by 

M(x,w, y,z)  »  sup  Inf  {<u,z>  +  <v,w>  +  K(u,x,vsy)}> 
u  v 

Then  M(x,w,y,z)  •  -W(-z,y,-w,x) ,  where  W  belongs  to  the  partial  conjugate 
of  [K]  In  u  and  v.  Hence  It  follows  from  Theorem  5.1  that  M  is  c-osed 
proper  concave-convex  and  depends  only  on  [X],  and  that  K  Is  polyhedral 
whenever  K  is  polyhedral.  The  equivalence  ciass  containing  M  is  called 
the  Lagrangian  of  S(K).  Similarly,  the  Lagrangian  cf  S(L)  is  the  equiva¬ 
lence  class  containing  the  function  N  given  by 

N(x,w,y,z)  *  sup  inf  (<s,x>  +  <t,y>  ♦  L(s,z,t,w)). 
t  s 

In  view  of  the  next  theorem,  [M]  is  called  the_  Lagrangian  of  the  dual  pair 
S( K) ,  S(L).  From  the  fact  that  the  partial  conjugacy  operation  induces  a 
symmetric  one-to-one  correspondence  among  closed  proper  equivalence  classes, 
it  follows  that  a  dual  pair  of  generalized  saddle  programs  is  completely  de¬ 
termined  by  its  Lagrangian. 

THEOREM  6.9.  The  saddle-functions  M  and  N  are  equivalent. 

PROOF.  Since  CL]  is  obtained  from  CK*'I  via  the  relation  l(s  ,z,t,w) 
-K*(-z,s,-w,t) ,  It  follows  by  (36.1)  and  Theorem  0.1(b)  that  [Lj  contains 
the  function  L  given  by 

L(s,z,t,w)  a  -  sup  inf  inf  sup  (<u,-z-  *  <x,s>  +  <v,-w>  +  <y,t>  -K(ij,x,v,y)} 
y  x  u  v 

=  -  sup  Inf  {<x,s>  +  <y,t>  -M(x,w,y,z)>. 

y  x 

If  [H]  denotes  the  partial  conjugate  of  [M]  in  x  and  y,  chi  s  mo  a  nr. 

[L]  a  l-H].  Now  by  Theorem  5.1  the  function  N  depends  only  on  CL]  Hence 
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N  is  equivalent  to  the  function  W  given  by 

N(x,w,y,z)  *  sup  Inf  {-s„x>  <t,.y>  -H{s ,z>t,w) } . 

-  « 

But  N  belongs  t.o  the  partis!  conjugate  of  fH]  In  s  and  t,  end  by 

fheorem  5.2  this  Is  the  same  as  [M].  This  shows  that  N  £  t'K'.i,  and  hence 

IN]  *  I'M]. 

The  next  theorem  says  that,  up  to  a  reordering  of  the  variables,  che 
conjugate  of  the  lagranglan  of  S(K)  coincides  with  both  toe  partial  conju¬ 
gate  of  [K]  in  x  and  y  and  the  partial  conjugate  of  til  in  z  and  w. 

THEOREM  6.IO.  Let  [M]  be  the  Lagranglan  of  $(K),  jet  H  be  any  mem- 
b&r_  of  the  partial  conjugate  qt  [ KJ  i r  x  and  y ,  arid  jet  J  be  any  member 

of  the  partial  conjugate  of  LU  jn  i  and  w.  Then  [M*]  cont_aj_PS  the 

functions 

(s,v,t,u)  ♦  H(v,s ,u,t) 
and 

(s,v,t,u)  i-  J(t,u,stv) » 

PROOF  Recall  that  [M]  contains  (x,w»y,z)  -W(  ,  where  W 

is  a  member  of  the  partial  conjugate  of  [K.3  'Mi  u  and  v.  Hence  (using 

(36.1)  and  Theorem  0. 1(b)  to  interchange  "sup"  vith  "inf'1),  [M*j  contains 

z  x 

(s,v,f,u)  sup  inf  sup  inf  /<x,s>  +  <w,v>  +  vy,t>  <z,u>  +  W(-z.y,»wvx)}. 
y  <  i  * 

How  observe  that  (u.x,v,y)  -*  inf  sup  {<•■/., u>  +  <-w,v>  *W(-z,y,-w,x)j  is  a 

:  w 

member  of  the  partial  conjugate  of  £Wj  in  z  and  w,  which  by  Theorem  5.2 

!>  just  [Kj.  Hence  M*  contains  is.v,t,u)  -v 

#♦-«'  ,  /*/ 

sup  inf  {<x,s>  +  <y.,v,>  K(u,x,v,y»)  for  some  Kc  [Kj.  But  up  to  reordering 

./  x 

the  variables  this  function  belongs  to  the  parciai  conjugate  of  r Kj  in  x 
and  y  This  shows  th.if  [MM  contains  (s,v,t,u)  -*  H(v,s  ,u,  t) .  Similarly, 
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where  r!(x,w,y,z)  -  sup  Inf  {<u,z>  ♦  <v,w>  +  K(u»x. v,y) }.  Since 

U  V  _  _  i 

(z.s.w.t)  *  sup  inf  Inf  sup  {<u,z>  +  <x,$>  +  <v,w>  +  <y,t>  -K(u,x,v,y)} 
y  x  u  v 

belongs  to  [K*j,  this  means  that  CM*]  contains 

;  .,nt,u)  ■*>  sup  Inf  {<z,u>  *  '.w,v>  ♦  K*(-z,s,-w,t)} 
z  w 

for  some  R*  t  l'K*]t  1.8. 

fs,v,t,u)  sup  inf  (<z,u>  *  <w,v^  -L($,z,t,w)} 
z  w 

for  some  l  c  [LJ.  But  up  to  reordering  the  variables  this  function  belongs 
to  the  partial  conjugate  of  f.U  In  z  and  w.  This  shows  that  [M-'j  con¬ 
tains  (s,v,t,u)  •*  J(t,u,s,v). 

COROLLARY  6.10.1.  If  the  saddle-value  of  the  Lagranqlan  exists  and  equals  i 

a,  where  a  c  R.  then  the  optimal  values  In  S(K)  and  S(L)  exist  and  equal 

a. 

PROOF.  Tne  saudle-value  of  M  exists  and  equals  a  Iff  H*(o,o,o,c)  - 
fHo.o.o.o)  *  -a.  By  Theorem  0.1(b)  this  ts  equivalent  to  ff*(o,o,o,o)  * 
for  every  M*  c  [,“*3,  for  1  ■  1  and  2,  -P^(u,v)  n  H^v.o.u.o)  for  a  cer¬ 
tain  member  Hj  of  the  partial  conjugate  of  CKJ  In  x  and  y,  and 
-Q.(s.t)  *  J^t.o.s.o)  for  a  certain  member  of  the  partial  conjugate 
of  tU  In  z  and  w.  Hence  Theorem  6.10  implies  that  -P^ ( o ,o)  -  -a 
-Qj(o,o)  3  -a  for  1  *1,2. 

COROLLARY  6.10.2.  If  there  exist  points  u  jind  v  such  that 
(o,v,o,u)  t  ri(dom  M*),  then  -p^  and.  -f^  belong  to  a  closed  proper  equiva¬ 
lence  class  which  contains  the  upper  and  1  owe r  conjugate  of  every  obja tlve 
function  of  S(L).  Dually,  if  there  exi»t  points  s  and  t  such  that 
r:. ,o,t,o)  e  ri(dom  M*),  then  -(^  anti  -Q?  belong  to  a  clc  d  £rop/.’r  o  jinva- 
lence  class  which  contains  the  upper  and  lower  oonjujjate  nt  •  very  object i ve 
function  of  S ( K) . 


80 

PROOF.  Assume  (o,v,o,u)  ■&  ri(<iom  M*)  for  some  u  and  v.  Then 
Theorem  6.2  Implies  that  the  functions  (v,u)  R*(o,v,o,u)  for  R*  e  L'M*'j 

all  belong  to  a  single  closed  proper  equivalence  class.  3y  Theorem  o.i*' 
this  Implies  that  the  functions 

(v,u)  -P,(u,v)  ■  Inf  sup  (<x,o*  +  <y,o>  -K(u(x.v,y)}( 

*  y 

(v,u)  ♦  -P?(u,v)  ■  sup  inf  { <x ,a>  +  <y,o>  -K(u,x, v,y)}, 

' _ y  x 

(v,u)  (r(o7* ,o7*T)*(u,v)  ■  inf  sup  {<z,u>  +  <«,v>  -t(o 

w  z 

(v,u)  (t(Of  ,o>« ) ) w(  u ,  v)  »  sup  Inf  {<z,u>  «•  <w,v>  -L(o,z,o,w) ) 

Z  W 

are  equivalent,  closed  and  proper.  The  dual  assertion  follows  similarly. 

We  call  the  conditions  in  the  next  lemma  the  extrema  1  i t.y  conditions 
associated  with  S ( K)  arid  5{L). 

LEMMA  6.11.  for  any  (x,y)  e  R1"  *  Rn  and  (z,w)  c  Rp  x  R\  the.  .[pjjpw_- 
iQS.  conditions  are  equivalent: 

(a)  ( -z,ij ,-w, o)  c  c'K.(o:.x.o,y) ; 

(b)  (-x,o,-y,o)  r.  cl(o,z,o,w); 

(c)  (o,o,o,o)  r  J>M(x,w,y,z}, 

(d)  (x,w ,y,z)  Is  a  saddle-point  of  the  lagranglan. 

PROOF.  Observe  that  ~M(x,~w, y,-z)  a  W(z,y,w,x),  where  W  is  in  the 
partial  conjugate  of  [K]  in  u  ami  v.  Also,  Theorem  5.3  implies  (a)  is 
equivalent  to  (o, 0.0,0)  t  aWf-z.y.-w,;*.).  By  (27.4)  it  follows  that  (■•;)  is 
equivalent  to  (c).  Trivic-.11/,  (c)  Is  equivalent  to  (d).  finally,  l.w.5) 
implies  that  (a)  is  equivalent  to  (o,x,ovy)  r  3K*(-z,o,-w,a).  ii.v  37. 4)  *nd 
the  relation  L(s,z,t,w)  -  -Y.*(  z,’. ,-w,t)  it  follow.^  that  this  l.v,»  cotiditioi 
is  equ.  /talent  to  (b) . 

THEOREM  6.1?.  Assume  U,.v)  ».  *  Rn  and  (z,w )  e  Rp  *  R1  sat: sty 

the  extremal ity  conditions.  Then  ( x , y )  is  an  optimal  solution  r.i'  i'.'d  ano 

(z,w)  js  a  KiihrH’ucf'er  vector  •?.!)  ^» ( K ) -  Dually,  (z.w)  is  an  optimal 


solution  of  S(L)  and  (x,y)  Is  a  Kuhn-Tucker  vector  tor  S(l.). 

PROOF.  8y  Lemma  6.11  we  can  suppose  that  (~z,o,-w,o)  e  3K(o,xfo,y) . 
Then  (37.4)  and  (37.4.1)  imply  that  all  the  members  of  CK J  have  the  same 
finite  value  a  at  (o,x,o,y),  and  moreover 

V(u.x'  ,o,y)  r  --I iti>  <  a  <  K(o,X,v,y')  ♦  <V,W» 
fur  all  (u,x‘)  c  Rp  *  P10,  (v,y‘)  e  R**  *  Rn  and  t  e  CIO.  Taking  u  n  o 
and  v  ■  o,  this  shows  (x.y)  is  a  saddle-point  of  K(o,*,o,*)  for  each 
K  c  CK] .  Hence  (x,y)  is  an  optimal  solution  of  S(K),  and  P,(o,o)  *  a  * 
Pgto.o).  Also,  by  taking  K  ■  K  we  obtain  <u,z>  +  sup  K(u,*,.o,y)  <  a 
for  all  u  c  Rp,  and  by  taking  ft  ■  K  v/e  obtain  a  <  Inf  K(o,x,v,0  <v„w> 

for  all  v  c  Rq.  Hence  (z,w)  Is  a  Kuhn-Tucker  vector  for  S( K) .  The  duel 

assertion  follows  similarly,  using  the  condition  (-x,o,-y,o)  r.  ai.(a,z.o,w) . 

The  next  result  is  a  generalization  of  the  Kuhn-Tucker  theorem,  U  is 
refined  somewhat  by  Corollaries  6.17.2  and  6.17.3, 

THEOREM  6.13.  Assume  S(K)  Is  strongly  consistent.  Then  the  pair 
(x.y)  X  Rm  x  Rft  fs  an  optimal  solution  of  S(K)  Iff  there  exists  a  gaj r 

(z,w)  c  Rp  x  such  that  (x,w,y,z)  Is  a  saddle-point  of  the  lagrangien. 

Such  a  pair  ( z ,w)  21 1  Kuhn-Tucker  vector  for  S ( K) . 

PROOF.  By  the  dual  version  of  Corollary  6.6.1,  (x,y)  Is  on  optimal  sol 
utlon  of  S(K)  iff  there  exists  a  pair  (z,w)  such  that,  (-x,o,-y ,r.)  t: 
3L(o,2,o,w).  The  theorem  follows  from  this  by  Theorem  6.12. 

By  analogy  with  (36,6)  for  convex  programming,  one  might  ask  whether  the 
last  assertion  of  Theorem  6.13  can  be  strengthened  to  the  following:  "Such 
pairs  ( z ,w)  are  precisely  the  Kuhn-Tucker  vectors  for  S(K)."  The  next 
example  demonstrates  that  this  does  not  hold  in  general.  The  reason  is  basic 
ahy  that  the  set  DM*(o,o,o,o)  of  saddle-points  of  the  Lagrungi.-ir.  is  f:c 
product  set 
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3M*(* ,* ,o,o)(o,o)  *  3M*(o,0,‘ ,-)(o,o) , 

in  which  each  "factor"  involves  both  the  pair  (x,y)  of  "solution  variables" 
and  the  pair  (z, w)  of  "KuhrHTucKer  variables." 

EXAMPLE  6.14.  Take  p  •*  n  and  q  «  m  and  define  S(K)  by 
K(u,x,v,y)  ■  <u,y>  ♦  <x,v>.  It  is  easily  checked  that  Aj*{0} A  (rec  cone^K)  ■ 
{(o.o))  for  1  •  1,2.  Hence  Lemma  2.9  Implies  that  range  A*Ar1(dom  K*)  f  (5, 
or  in  other  words  S(L)  is  strongly  consistent  (see  the  proof  of  Lemma  6.41. 

If  [P]  denotes  the  equivalence  class  containing  P1  and  P^,  then  LP‘J  - 
LAK]  by  Theorem  6.5.  Since 

/  o  If  u  •  a  and  v  *  o 

P^(u,v)  ■  <  +•  If  u  •  o  and  v  f  o 

^  If  u  t  o 

this  Implies  that  dom  AK  «  dom  P  ■  {{o.o)},  whereas  A  dom  K  »  Rp  *  Rq. 

Clearly  S(K)  is  strongly  consistent,  the  set  of  optimal  solutions  of  S(K) 

is  R1”  *  Rn,  and  the  set  of  Kuhn-Tucker  vectors  for  S(K)  is  Rp  *  Rl  The 

Lagrangian  of  S(K)  contains  the  function 

M(x,w,y,z)  -  sup  inf  {<u,z>  +  <v,w>  +  K(u,x,v,y)} 
u  v 

✓  o  if  x  +  w  *  o  and  y  +  z  n  o 

if  x  +  w»o  and  y  +  z/  o 

^  —  If  x  +  wfo 

Hence  the  set  of  saddle-points  of  the  Lagrangian  Is  just  dom  M  « 

{ ( x ,w) | x  ♦  w  «  0}  x  {(y,z)|y  +  z  *  o).  Thus,  If  (z.w)  is  any  given  Kuhn- 
Tucker  vector  for  S(K),  the  set 

((x,y)|(x,w,y.z)  Is  a  saddle-point  of  the  Lagrangian} 
equals  {(-w,-z)}  and  hence  Is  far  from  the  equalling  the  set  of  optimal  solu¬ 
tions  of  S(K).  It  Is  of  Interest  to  note  that  the  dual  program  S(i.)  is 
given  by  l.(svz,t,w)  *  <s,w>  •*  «- z.t>  and  hence  Is  "identical"  with  S(K). 
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In  order  to  describe  more  fully  the  duality  between  the  optimal  solutions 

of  S(K)  and  5(1)*  we  Introduce  another  definition.  Tor  each  x.  in  sf* 

define  the  function  f  on  R**  by  fx(v)  ■  Inf  K(olx*v»«)>  and  for  each  y 

In  Rn  define  the  function  g  on  Rp  by  g  (u)  ■  sup  iT(u»*  ,o»y).  It  fo<- 

¥  «/ 

lows  easily  from  (5.7)  that  f  is  convex  and  g  Is  concave.  An  optimal 

x  y 

solution  (x,y)  of  S(K)  Is  said  to  be  stable  iff  the  directional  derivative 
function 

v  +  fi(osv)  -  lim  x'^fjxv)  *  f  (o)) 

*  AH!  x 

Is  never  -»  and  the  directional  derivative  function 

u  9^(oits)  *  Hm  (An)  -  g  (o)) 
y  Aio  y  y 

Is  never  -*•».  It  is  not  hard  to  show  that  (x,y)  is  an  optimal  solution  cf" 
S(K)  Iff  fx(o)  *  gy(o)  «  R.  Hence  by  (23.1)  the  directional  derivatives 
mentioned  in  the  definition  exist  (+»  and  -<*  being  allowed  as  limit:;:}. 
Stable  optimal  solutions  of  S(L)  are  defined  similarly,  using  the  functions 
hw(s)  »  Inf  L(s,',o*w)  and  k2(t)  ■  sup 'f(o,z*t»*). 

LEMMA  6 A 5,  Let  (x,y)  be  an  optimal  solution  of  S(K),  Thesi  (x,y)  v 
stable  Iff  f  and  9  are  subdlfferentiable  at.  the  origin,. 

PROOF.  By  (23.2)  and  (23.3), 

LEMMA  6,16,  For  (x„y)  c  R^  *  Rn  and  (z,w)  e  ,  each  cl  .the 

following  condl  Huns  is  equivalent  to.  the  extrema  lily  conditions  in  Lemma 
6JJ: 

(e)  ■•(*,«)  e  ag^o)  x  afx(o)  and  fx(o)  *  9y(o)  c  R; 

(f)  ~(x,y)  r.  ahw(o)  «  2kz(o)  .and  hw(c)  ■  ^(0)  e  R. 

PROOF.  By  (37.4)  and  (37.4.1),  (•■?., o,~w,o)  e  e>K{o,x,o,y)  occurs  iff 
K(o,x,o,y)  3  K(0:Xfco,y)  -  u  e  R 


and 


<u,z>  *  f(u,x,o,y)  <  a  <  K(o)x,v,y)  ■>  <vtw> 
for  all  u  e  R?,  x  c  ft*,  v  *  and  y  t  Rn.  But  this  occurs  iff 

0y(o)  *  fx(0)  *  »  e  ft 


<U,<>  ( u)  <  a  <  fjv)  +  <v,w> 

for  all  ut^  and  v  e  1  his  last  condition  holds  Iff  (t.)  holds,  Simi¬ 
larly,,  (~xto,-y»o)  t  sLtc.Zj.o.w)  occurs  Iff  ( 1)  holes. 

TKEORKK  6,17.  The  pair  (x,y)  Is  a  stable  optimal  solution  of  S(X) 

Iff  (“Z,or-w,o)  e  3K(o,xfto,y)  for  sorts  pair  (z»w). 

PROOF.  By  Lemma  C.15s  (y.,y)  is  a  stable  optimal  solution  of  $;>'.)  iff 

f  Jo)  *  g  (o)  t  ft  and  ag  Jo}  *  3f  Jo)  is  nonempty.  New  apply  imwi  8.15. 
x  y  •  y  x 

COROLLARY  5.17.1.  The  :  "*qra>n  S(K)  has  a  stabia  optimal  sel'.rhw  iff 
S ( L )  does,  Iji  which  case  the  two  optimal  values  are  equal , 

PROOF.  Apply  the  theorem  to  both  S(K)  and  S(L)  and  use  Law*  ft.;'!, 
The  two  optimal  values  are  equal  by  Corollary  6.10,1. 

COROLLARY  6.17. i.  If  $(K)  Is  strongly  consistent,  then  every;  Mti&l 
solution  of  S(K)  is  stable. 

PROOF.  .Suppose  S(K)  is  strongly  consistent,  end  let  (x,y)  oe  <wy 
optimal  solution  of  S(K).  By  the  dual  version  of  Corollary  5.5.1,  I  here 
exist  points  /.  and  w  suc?i  that  (••x,o3-y,o)  e  ?L(u,?.,o,w)  Kenw  U*m 
6.11  anc  the  theorem  Imply  (x,y)  is  stable- 

COROLLARY  6.17.3.  £  pair  (x,y)  is  a  staple  optimal  soli  t Ki  of  .${;<} 
jiff  the iti  exists  a  fair  (zfw)  such  that  (x,«r,y,z)  is  a  sadol e-sui.it  of  tne 
Laqranglan,  and  such  £  jgair  ( z,w)  1_s  a  Luhn-Tuckt-r  vector  fC  "  J  l,  K  ‘  . 

PROOF.  By  Theorem  G .17.  L smses  6.(1  and  Theorsn.  6.1?. 

COROLLARY  6.!?  4.  Assume  that  the  optimal  v^lue  in  p-h-.  ^ 

equals,  a.  If  either 
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11m  i"1(P|(o>xv)  -  o)  *  -* 

X-fO 

for  some  v  or 

mm»«  aiOTMMM  mmam 

11m  x“1(P2(Xu»o)  -  o)  •  +• 
x+o 

for  some  u,  then  neither  ${K)  nor  S(L)  has  a  stable  optimal  solution. 

PROOF.  Suppose  (x,y)  Is  an  optimal  solution  of  S(K).  Then  f-(o)  ■ 
g-(o)  e  R.  Notice  that 

<  sup  fx  *  P|(o,*)  and  P2(*»o)  «  Inf  gy  <  g-. 

Hence  the  hypothesis  Implies  either  f;(o;v)  *  -•»  for  some  v  or  g£(o;u)  * 
+•  for  some  u.  This  means  that  (x,y)  Is  not  stable.  Thus  $(K)  has  no 
stable  optimal  solution,  and  by  Corollary  6.17.1  neither  does  S(L). 

According  to  Corollary  6.17.2,  If  a  program  is  strongly  consistent  then 
all  of  Its  optimal  solutions  are  stable.  The  next  example  shows  that  In  the 
absence  of  strong  consistency  there  may  exist  unstable  optimal  solutions. 

EXAMPLE  6.18.  In  Example  6.3  take  a  «  n  ■  1  and  take  [J]  to  be 
such  that  [J*]  contains  the  closed  proper  concave-convex  function 

if  sc  fo,l]  and  t  t  [o,l] 

($,t)  i  If  st  [o,l]  and  t  i  [o,l] 
u  -»  if  s  i  [o,l] 

Clearly  dom  J*  *  [o»l]  *  [0,13,  J*(o,o)  *  J*(o,o),  and  aJ*(o,o)  *  p.  From 

i 

the  analysis  in  Example  6.3  It  follows  that  SfL)  is  consistent  but  fails  to 

i 

be  strongly  consistent,  S(L)  has  a  well-defined  primal  problem,  and  (o,o)  is 

i 

the  only  optimal  solution  of  S{1).  If  (o,o)  were  stable,  then  by  Corollary 
6.17.1  there  would  exist  a  stable  optimal  solution  of  S(K).  But  the  set  of 
optimal  solutions  of  $(K)  Is  easily  seen  to  be  3J*(o,o),  which  is  empty. 

* 

Hence  (o,o)  Is  unstable. 

By  Corollary  6.5.1,  If  the  dual  program  S(L)  is  strongly  consistent  then 

* 

each  Kuhn-Tucker  vector  (z,w)  for  S(K)  "corresponds"  to  a  saddle-point  of 
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the  Lagrangian  In  the  sense  that  (o, 0,0,0)  t  3M(x.w,y,z)  for  some  pair 
(x.y).  Example  6.18  shows  that  this  need  not  be  true  when  S(L)  falls  to 

be  strongly  consistent. 


f 7 :  Ordinary  Saddle  Programs  and  Lagrange  Multipliers 

Suppose  that  S  and  T  are  nonempty  convex  subsets  of  Rm  end  Rn, 
respectively,  and  that  H  Is  a  finite  concave-convex  function  on  S  x  1, 
g^,...,gp  are  finite  concave  functions  on  S,  and  are  finite 

convex  functions  on  T.  Consider  the  problem  of  finding  the  saddle -points  of 
H  with  respect  to  the  pairs  (x,y)  in  S  *  T  satisfying  the  constraints 

g1  (x)  >o,  i  «  i,...»p 
and 

fj(y)<o.  j  •  l . o. 

Under  suitable  regularity  assumptions  this  problem  can  be  cast  in  the  form  of 
a  generalized  saddle  program  of  a  certain  type. 

Let  H  be  a  closed  proper  concave-convex  function  on  *  Rn,  for 
1  8  let  be  a  closed  proper  concave  function  on  R^  such  that 

dom^H  c  dom  and  rl(dom^H)  C  ri(dom  g^)t 
and  for  j  «  let  fj  be  a  closed  proper  convex  function  on  Rn  such 

that 

dom2H  c  dom  f j  and  ri (dom^H)  C  ri(domfj). 

Let  subsets  C  c  Rp  x  r"1  and  0  C  Rq  *  Rn  be  defined  by 

C  *  { (u, x) j x  e  dom^H  and  g^(x)  >  u^  for  1  =  p) 

and 

0  “  ((v,y)|y  e  doni^H  and  f^(y)  <,  Vj  for  j  •  1 . q>, 

and  define  a  function  K  on  (Rp  *  R^)  x  (Rq  x  Rn)  by 

/H{x,y)  If  (u,x)  c  C  and  (v,y)  c  D 

K(u,x,v,y)  »  |  if  (u,x)  e  C  and  (v,y)  i  0 

—  if  (u,x)  i  C 

THEOREM  7.1.  The  function  K  _1_s  closed  proper  concave-convex  with  do- 
mein  C  x  0.  Moreover. 

ri  C  *  {{u,x))x  e  rl(dan^H)  and  g.(x)  ;  for  1*1 . p) 
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and 

cl  C  ■  { (u»x)i x  r.  cl(do«.|H)  and  <jj(x)  >  u^  for  1  •  1 . p}( 

and  similar  formulas  hold  for  rl  0  and  cl  0. 

PROOF.  Define  functions  Hq . Hp+q  on  (Rp  *  Rm)  x  (Rq  x  Rn)  as 

follows: 

H0(u,x,v,y)  «  H(x,y) 


Hi(u,x,v,y)  • 

(0 

if 

(x,^)  t  epl  ^  ^ 

(x,u1)  t  epl  9^ 

L 

if 

ptj(u,*.v.y)  •  ■ 

r° 

if 

(y.v,)  t  epi  f, 

3  3  J  -  1 

(y.Vj)  i  epi  fj 

L 

if 

Clearly, 

dom  Ho  ■  (Rp  *  dom^H)  x  (Rq  x  dom^H) 

dom  Hj  »  {(u,x)|(x,u<)  e  epl  g^}  x  (Rq  x  Rn)  1  ■  l,.  ,.,p 

dom  H(  j  ■  (Rp  x  rP)  x  {(v.y)|(y,Vj)  c  epl  fj>  j  ■  1 . q 

and  from  (34.3)  It  follows  that  each  Is  closed  and  proper.  Since 

r1(dom  H  )A***nr1(dom  Hp+<J)  f  d. 

Theorem  4.2  Implies  that  CHg3  ♦  ...♦  [H^]  Is  well-defined,  has  domain 

C  x  D  *  dom  HaA. . .  Adorn  H^, 

and  contains  the  function  K.  The  formulas  for  ri  C  and  cl  C  (resp.  rl  0 
and  cl  D)  follow  from  (6.5),  (7.3)  end  the  fact  that  epi  (resp.  epl  fj) 
Is  closed. 

According  to  the  theorem,  S(K)  Is  a  generalized  saddle  program  on 

R1*  x  Rn  with  perturbations  in  Rp  x  Rq.  He  call  S( K)  the  ordinary  saddle 

program  associated  with  H,  g^,...,Sp,  t^,...,f^. 

It  will  be  convenient  to  int.-oduce  the  following  notation.  For  any  sub¬ 
set  S  of  Rp  x  r"1  write  Su  *  t  S/  for  eocn  u  c  Kp.  Similarly, 

for  any  subset  T  of  Rq  x  Rn  write  T  *  iy|(v,y)  c  T)  for  each  v  e  R°‘ 

V 
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Since  the  feasible  solutions  of  any  generalized  saddle  program  are  those 
pairs  (x,y)  such  that  (o.x.o.y)  c  dom  K,  the  set  of  feasible  solutions  of 
the  ordinary  saddle  program  $(K)  Is  just  Cq  *  Oq,  l.e. 

{(x,y)  e  dom  Hjg^x)  >.  o,..  .,gp(x)  >.  o  and  fj(y)  <  (y)  <  o). 

Recall  from  the  general  theory  that  S(K)  Is  consistent  iff  S(K)  has  a 
feasible  solution,  l.e.  Iff  C<>  *  Do  Is  nonempty. 

COROLLARY  7.1.1.  The  program  S(K)  i&  strongly  consistent  Iff  there 
exists  a  pair  (x*y)  in  r1(dom  H)  such  that  g^(x)  >  0,...,gfJ(x)  >  o  and 
fj(y)  <  o,...,fq(y)  <  o.  Moreover,  this  equivalence  still  holds  If  'Vi"  Is. 
deleted. 

PROOF.  The  original  equivalence  assertion  Is  Immediate  from  the  formulas 
for  rl  C  and  rl  0  given  by  Theorem  7.1.  Now  suppose  (x.y)  c  dom  H  is 
such  that  g^x)  >  o,...,gp(x)  >  o  and  f^y)  <  o,.  .  ,fq(y)  <  o  Let  (x^.y.) 
be  ar^y  element  of  r1(dom  H).  Then  (6.1)  and  (7.5)  imply  that,  for  sufficient¬ 
ly  small  positive  x,  the  pair 

( \>yx )  *  (1  -  *)(x,y)  +  x(x,,y,) 

Is  in  r1(dom  H)  and  satisfies  9j(xx)  >  o . gp{xA)  >  o  and 

fj(yx)  <  o9...,fq(yx)  <  o. 

If  $(K)  Is  strongly  consistent,  then  all  the  objective  functions  of 
S{K)  are  equivalent  and  hence  the  notions  of  optimal  value  and  optimal  solu¬ 
tion  can  be  expressed  In  terms  of  the  single  objective  function 

rH(x,y)  if  x  c  Co  ard  y  r.  D0 

K(o,x,o,y)  •  <  +*  if  x  e  C0  and  y  £  Da 

^  if  x  £  Co 

In  this  event,  the  optimal  value  in  S(K)  exists  and  equals  a  iff 

sup  Inf. H  ■  inf  sup  K  *  a  e  R, 

Co  Oq  Dq  Cq 

ana*  (x.y)  is  an  optimal  solution  of  $(K)  iff  ( x,y )  is  a  saddle-point  of 
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H  with  respect  to  C0  *  0q«  The  characterizations  In  the  following  corollary 

hold  even  when  S(K)  Is  not  strongly  consistent. 

COROLLARY  7.1.2.  Write  clC“  C*  and  cl  D  a  D' .  The  optimal  value 

jn  S(K)  exists  and  equals  a  iff 

sup  Inf  H  ■  Inf  sup  JT  «  a  c  R, 

01  ~  Ob  «  . 

A  pair  (zsw)  Is  a  Kuhn-Tucker  vector  for  S(K)  iff  the  optimal  value  In 
S(K)  exists  and  equals  3  and 

<u,r>  +  Inf  sup  IT  <_  a  <  sup  Inf  H  +  Vv  Vw. 

cb  si  ^ 

A  pair  (xty)  Is  an  optima  1  solution  of  5(K)  Iff 

inf  r;(x,‘)  *»  sup  H(*  »y)  t  R 

%  ~  Of 

An  optimal  solution  (x^y)  of  ${K)  is  stable  Iff 

11m  A“l(inf  K(x,» )  -  1nfH(x,*))>  -•»  Vv 
x*o  0;v  * 


and 

11m  A-i(s«p  ff(*  »y)  -  supFT(",y))  <  +»,  ’/u, 

uo  CVj  Q 

PROOF.  By  Theorems  7.1  and  0.1,  the  least  rraiber  K  of  L Kj 
convex  closure  of  K.  Direct  computation  thus  yields 

/H(x,y)  If  (u,x)  t  C  and  (v,y)  t  O' 

K(u,x,v,y)  »  \  +»  if  (u,x)  c  C  and  (v,y)  /t  D* 

“»  It  (u,x)  A  C 


Is  the 


and  hence 


>,(u,v)  *  sup  Inf  H. 

1  CD'" 

u  v 


Analogous  formulas  hold  for  R  and 
tor  for  S(K)  Iff  Pj(o,o)  *  F^o.o)  * 
P,(o,v)  +  <v,w>,VuVv.  Since  Mg,<j) 

I  1 


A  pair  ( 1  ,w)  is  a  Kuiw-fucker 
a  c  R  and  <u,z>  ♦  F..{u ..■;»)  ^  a  _ 
a  P^(c,d}  “Cfj  oci.tr-  \  "  tvt 
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optical  value  in  S(K)  exist*  arc'  V,  o.  the  first  two  assertion1,  of 

the  corollary  follow  immediately  front  the  formulas  for  P1  and  P;) .  Now 

recall  from  §6  the  functions  f  ( v)  «  inf  K(o,x,v,*)  and  g  (u)  - 

*  / 

sup  K(u,* ,o,y) .  The  formulas  for  K  and  K  imply  that 

?,{v)  »  inf  il< x , * ) 


whenever  x  e  Co  and 


g  (u)  *  sup  fT(  •  ,y) 
y  r » 


whenever  y  e  Oq.  Since  a  pair  (x,y)  is  an  optimal  solution  of  S *, K >  iff 

f  { o l  3  g  (e)  k  R  fend  (y.x)  f.  U>  *  } ,  the  third  assertion  follows  inne- 

x;  -y 

Jlately.  The  last  assertion  cUo  fellows  immediately  f**om  the  formulas  for 
fx  ani  V 

All  the  general  theory  of  $6  can  o?  applied  to  the  ordinary  saddle  pre- 
gram  $(K).  However  v.e  shall  deal  only  with  the  cu.est  esi  of  whether  there 
exists  a  good  Lagrange  m.uH’plier  principle  for  S(K) .  As  a  first  stop,  the 
next  results  identify  the  Ixgrangian  and  the  extremal Ity  conditions  associated 
with  $(K). 

THEOREM  7.2.  The  tagrangi^n  of  ${K)  contains  the  function 

yH(x,y)  t  I'c.g.ix)  +  <*- v**j f ^ 'y)  if  (x,w)  c  5  and  (y.z)  c  f 

lr  ;x(w)  c  6  and  (>,i)  t  T 
if  (x.w)  /  S, 

where  5  «=  dom.H  x  and  T  =  ucrru!!  >  R1^. 

PROOF.  By  definition,  the  Lagr-anqian  corf3 ins  the  function 

M(x,w,y,:}  s  sup  inf  {-u,z^  +  -  y ,w>  ♦  Kfj.x.v  ,y)} 
o  v 

*•  sip  inf  (<u,z>  +  <.\w-  +  H(x,y)}, 

•  ♦ 

V-  t  • 

X  ) 

C  t  { u j  ( u , x )  r  0}  and  0  3  'vi(v.y)  t  D) .  how  C  equals 
(x  >  u,,.. .  ,a  (k)  >  u  }  when  *.  Jon.li  anr  equals  the  empty  -;nt 

'  —  I  '  p.  i 


i 

(x,w„y,z)  ->  { 


{ 


who  re 


c,> 


mer-wlse,  ,.n  .  ■,  i  n  < a r » y  (J 


',Udf  1 


ly) 


j  f  when 


y  c  do*r,  t  <jnc:  equftSs  the  ei-.pt>  set  otherwise.  Therefore  the  confer  tiers 
inply  that  m*.,w,y.z)  *  —  when  t  t  and  M(x,wo,z)  *  <->  when 

.  i.  dort(ll  and  y  r  dor,9M.  When  (x,y)  i  dor.!  If, 

M(x,w,y,z)  -  H(x,y)  +  sup  {^u,z>  ♦  In*  ••'■v, v>)} 

r- 

i  . 

y 


s  —  if  w  t  Rj 

••  s  f»  If  w  t  and  z  i 

^H(x.y)  ♦  T’.Zj'M/j  ♦  Zwjf/y)  if  w  t  and  z  c 

it  is  ea<y  to  s-,o*  that  M  *•  c!-M  is  given  by 

.  H(x,y)  +  lz  | y 4 ( x )  »•  £w<f,{.yl  if  (x,w)  t-  j  and  {/,:!  t  rl 

M(x,v?,y.z)  ■  ^  *«  if  (x,w)  t  $  and  (y,i)  t  :1 

If  (*.w)  t  S 

MnaMy,  observe  that  tee  fih.tlon  in  the  theo-erc  is  bounded  below  by  M  and 
•bOVti  by  M. 


COROLi.ARi  7.^.1.  Two  £Mys_  (>,y)  an_d  (z.w)  satisfy  the  e:.t  remall  ty 
i.thcrs  dS.soc_l_ated  with  >(<)  iff 


l  ^  ij 

.)  t 

co  ' 

D0,  U  »w;  l  * 

',q- 

(x) 

*  0 

fer  1  *  1 . p, 

Vj 

;(>) 

*  0 

for  ,i  ■  1 _ 

0  C 

,*.y) 

■  i  1  a  | )  ( x) . 

0  t 

a^H  ( X .  y ) 

-  Ei', "/<)(>). 

The  term  Za(z^y()(x;  ijn  'e  reduced  by  where  the  r.u.nriati  on 

e .^er^ s  .inly  over  those  I  oct  that  i ^  »  o.  Similarly .  the  term 
Xtf(w,f.)(y)  nar_  tie  replaced  *jy  T.w  ?f.(y),  where  the  sjzrona  r  i  c  *i  extends 
only  ove-*  those  j  such  that  - .  c . 

P  (OOF.  [)y  definition,  'x.yf  j-1  (z,w)  satisfy  the  ext'enai.t  ton 


<iit ions  iff  (x,w,w,z)  1:.  a 


'•  :"j;nt  of  tot*  !  a  ’ranu '  an .  By  *o»  theorem 
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optimal  value  in  MM  ‘’xist*  °\  n  .  t.  >e  firsr  two  a  .«/-»■  1 1  o.',  of 

the  corollary  follow  Immediately  f rwi  t'e  formulas  fo-  Rj  an  I  !\, .  Now 

re^a'I  from  IG  the  f  motions  f  (v)  **  Inf  K/o,>,v,*)  and  g  (u)  » 

x  y 

sup  Mu.-.o.y).  The  formulas  for  K  and  K  Imply  that 

fr{v)  «  Inf  H(x.* ) 

,  » 

whenever  x  i  Co  and 

g  (o)  *  sup  fT(  •  ,y) 

Cu 

whenever  y  t  Ur, .  Since  a  pair  (a  y)  Is  an  optimal  solution  of  S’M  Iff 

f .  ( o )  ■*  y  (c)  ►.  R  (anil  (y,x)  s  c<j  *  ) ,  the  tKi  rd  assertion  follows  wire 

x  y 

Jiately.  The  last  assertion  a'so  follows  Immediately  Mon  the  formulas  for 

f  and  q  . 
x  -y 

All  the  general  theory  of  16  can  be  applied  to  the  ordinary  saddle  pre- 
gram  5(K).  However  w*»  shall  deal  only  with  the  ci.ost. t.i  of  whether  triers 
exists  a  good  Lagrange  nuH’pHer  principle  for  MM  As  a  first  st-p,  tf..» 
next  results  Identify  the  Ingeap.-M an  ar-d  thr  extremal  1  ty  conditions  associated 
wit,.  S(K). 

THEORFM  7.?.  The  lagrangian  of  S ( K)  contains  the  funst i on 

/  H ( x , y )  ♦  -Z^(x)  +  *r  (x,w)  c  ^  <intS  M'.z)  •-  • 

(x.w.y.z)  >  '  +~  ir  (x,w)  t  P  and  (>,z)  t  T 

^  —  H  (x.w)  /  S, 

where  S  «=  dom,H  «  and  T  *  ooiryi  ■»  R*^. 

PROOF.  By  definition,  the  Ugrjngian  cor  tains  the  function 

M(x,w,y,:)  «  sup  inf  {<u,z*  +  •v.w-  ♦  Klu.x  ,s  ,y) } 
v  v 

*  sip  ir»f  (<u,z>  +  </,w,  i  H(x,y)>, 

:> 

*  a 

where  Cy  ~  {u|(u,x)  r  C)  and  u  J  lvj(v,y)  t.  D) .  Now  equals 

•;uig,(*l  >  u, , . .  .  ,q  (»1  >  u  )  whtn  •  Jom.fl  anr  equals  the  eruty  .et 

-  I  p  jj  i 
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and  (36.3),  this  occurs  Iff  (r.y)  *  dom  H  and  (z,w)  t  K  R^ , 

M( x '  ,y)  ♦  ^2igi (x ‘ )  ♦  Ewjfj(y)  <_  H(x,y)  f  Xz^fx)  v  Wjf  .(y)  ( 1 } 

for  all  (x\w*)  t  dom^H  *  P^,  and 

K(x.y)  ♦  Xz^U)  ♦  rw^fj(y)  «.  H(x.y')  *  Xzjg^x)  ♦  Xwjf^y')  (2) 
for  all  (y',z')  c  dom^H  *  R?.  Taking  z'  ■  z  In  (2)  and  using  (23  8)  Im¬ 
plies  o  c  ♦  X>(wjfj)(y).  Taking  y*  ■  y,  zj  -  1  +  z^  and 

z‘  •  zk  for  k‘  f  1 ,  (2)  implies  that  o  <_  g^(x).  This  holds  for  each  1. 

But  taking  y‘  ■  y  and  z*  ■  o  In  (2)  Implies  Xt^g^(x)  <_  o.  Hence 
*j9j(x)  "  o  Tor  each  1.  Similarly,  (1)  Implies  that  f  1  (y)  *^o  and 
Wjfj(y)  «  o  for  each  J  and  o  c  ^H(x,y)  2>(z^g^)(x).  This  establisnes 

one  Implication,  and  the  converse  is  now  clear.  Now  observe  that  w  >  o 

J 

trivially  Implies  a(wjfj)(y)  ■  Wyjfj(y).  On  the  other  hand,  If  ■  c  then 
y  c  don^H  C.  dom  fj  Implies  a(Wjfj)(y)  •  a«(y|rfom  f<)  c  36 ( y ]  do.r^H)  » 

,.y)  and  hence  a,H(x,y)  +  3(w^f^)(y)  -  a^Hjx.y).  Thus  the  term 
Xa(Wjfj)(y)  can  be  replaced  as  indicated.  The  other  assertion  is  Droved 
similarly. 

Variables  of  the  sort  z,,...,7  and  appearing  in  the  La- 

•  p  *  g 

granglan  of  S ( K)  are  known  tradi tionel ly  as  Lagrange  multipliers.  Sometimes 
this  term  also  denotes  the  particular  values  of  these  variables  which  satisfy 
certain  "extremal  1 ty  conditions”  relating  to  a  "lagrangian  function. "  In  this 

second  sense,  Lagrange  multipliers  ,  Wj . w^}  =  (z,w)  for  an 

ordinary  or  generalized  saddle  program  necessarily  form  a  Kuhn-Tucker  vector 
for  the  program  (Theorem  6.12).  However,  a  Kuhn-Tucker  vector  need  not  satis¬ 
fy  the  extremallty  conditions,  i..?.  need  not  b-j  a  Lagrange  mu’  tipi  ier.  (This 

behavior  can  occur  if  the  dual  program  fails  to  be  strongly  consistent.  See 
the  remarks  following  Example  6.18.)  In  other  words,  Kuhn-Tucker  vectors  i'-e 
defined  even  when  the  extranality  conditions  are  not  satisfiable.  Thus,  Kuh.n- 
lucker  vectors  (rather  than  Lagrange  multipliers)  ire  the  natural  'Vcui  1  ibr ium 
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pr’ce  sector*,"  fo  i  •?  j  ’  ar  t  '•  c  .('(4dl“-pol  nt  r>rnMc»rs. 

By  the  general  theory  of  >6,  If  (x,y)  and  U,w)  satisfy  the  extrem¬ 
al  Ity  conditions  then  the  optimal  value  In  S(X)  exists  and  equals  rl(x,y), 
(x,y)  Is  a  stable  optimal  solution  of  S(K)>  and  (z,w)  Is  a  Kuho-Tjcker 
vector  for  S{K).  In  fact,  such  pairs  (x,y)  and  (z,w)  actually  satisfy 
<u,z>  ♦  ff(  i'  ./)  <_  H(  x,y)  <  H{x,y‘)  ♦  <v,w> 
for  every  (u,x')  c  cl  C  and  every  (v  y')  t  cl  0.  (cf.  Corollary  7.I.?) 

The  next  theorem  Is  the  main  existence  result. 

THEOREM  7.3.  XI  S{Y.)  Xi  Strongly  consistent,  then  thf  extrema! It/ 

conditions  can  be  satisfied  whenever  the  sets 

P 

(x  t  A  rec  cone  g.jlnf  {i'ec  H(- ,y)(x) I (o,y)  t  rl  0)  -*  o} 

1-1  1 

and 

Q 

(y  e  n  rec  cone  fjsup  {rec  H(x,*  )(y) I (o.x)  t  rl  C}  <  o) 

J-l  J 

are  closed  under  scaler  multiplication  by  - 1 . 

PROOF.  By  Corollary  6.17.2  and  Theorem  6.17,  If  S(K)  Is  strongly  con¬ 
sistent  and  h3s  an  optimal  solution  then  the  extremallty  conditions  can  be 
satisfied.  The  remainder  of  the  proof  consists  of  showing  that  Corollary 
6.2.1  applies  to  yield  an  optimal  solution.  Suppose  $(K)  Is  strongly  con¬ 
sistent.  By  Theorem  5.2,  S(K)  has  a  well-defined  primal  problem  which  Is 
given  by  the  ciosed  proper  equivalence  class  [Ko3.  Moreover,  dom  Kq  - 
Co  *  ,  ri(dom  *o)  ■  (rl  C)0  *  (rl  C)c  ,  and  [Kg]  contains  the  function 

s  H(x,.y)  if  x  r.  Cc  and  y  e  Do 

K  (x,y)  *  )  *•  If  x  c  C0  and  y  f  Dq 

^  >  1  f  x  r  C 

Let  Y  ■  (y  J  f -j  (y)  <  o,...,f  (y)  ^ol,  Then  D0  r  Y  ndor^H,  so  that  Ko(**’)  * 
H ( x , * )  +  a( - 1 Y)  whenever  /  c  rifdom^H).  It  fellows  from  the  definitions  and 
(9.3)  that 


95 


(rec2*o)(y)  *  sup  (rec.  H(x,*)*'y)  ♦  rec  «(  •  !Y)(y)j(o,x)  >  rl  C}. 

♦  ♦  1 

Now  note  that  I'ec  fl(*|Y)  •»  <i{*Jo  Y)  by  (8.5),  and  o  Y  ■  C\  rec  cane  f. 
by  (8.3.3)  and  (8.7).  Those  facts  together  Imply  that  (rec2K0)(y)  <  o  Hr' 

q 

y  c  /^\  rcc  cone  f.  and  sup  {rec  H(x,*)(y)|(o.x)  c  rl  C)  <  o.  A  similar 

J"1  J  P 

argument  shows  that  (rec.Ko)(x)  >  o  Iff  x  r  f  \  rec  cone  5.  and 

1  v-  !  1 

Inf  (rec  H(*  ,y)(x)  |(o,y)  e  rl  0 )  ^  0.  These  two  equivalences  show  that  tr.^ 
hypothesis  Is  just  what  is  needed  to  apply  Corollary  6.2.1. 

For  each  (z,w)  e  define  a  function  H„  on  Rn  *  rtn  o.v 

{H(x,y)  ♦  X z | ( x)  ♦  Swjfj(y)  If  x  e  don^H  and  y  c  domyl 

*•  If  x  c  dom^H  and  y  i  dorrv.H 

-•  If  \  i  doW|H 

By  Theorem  4.2  It  follows  easily  froa  the  blanket  regularity  assumptions 

that  H.,  ^  Is  closed  and  proper  and  has  the  same  domain  as  H.  Observe 
x,w 

that  If  M  denotes  tne  Lagrmgian  given  in  the  statement,  of  Theorem  7.2, 

Hj^(A.y)  "  M(x,w,y,x) 

for  every  (z,w)  c  and  every  (x,y)  c  Rm  *  Rn.  If  (z,w)  ^  »  (ij 

put  $  *  fit  «nd  If  (z,w)  e  R^  >■  let  i  denote  the  set  of  pairs 

(x.y)  which  are  saddle-points  of  H  and  which  satisfy  the  conditions 

I  *  if 

g I ( x)  >  0  and  z^g^(x)  »  0  for  1  *  1 . p 


and 


fj(y)  10  and  vfjfj(y)  -  0  for  j  - 

(These  conditions  together  with  the  condition  (z,w)  e  x  are  tradition¬ 
ally  called  complementary  slackness  conditions.) 

For  ordinary  convex  programs  there  exists  a  good  Lagrange  multiplier  prin¬ 
ciple  (Theorem  28.1  In  (44]).  The  analogous  result  for  ordinary  saddle  pro¬ 
grams  would  be  the  following:  “If  (z.w)  is  a  Kuhn-Tucker  vector  for  S(t) , 
then  S  is  precisely  the  set  of  optima!  solutions  of  S ( K) . "  However,  the 

Z  |W 
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situation  Is  In  gentral  «or<  cwnpllcated  this. 

U.*ttA  7.4.  Two  pair;  (x,y)  and  (z,w)  satisfy  the  extremal Ity  condi¬ 
tions  1JT  (x,y)  c  $ 

I  |W 

PROOF.  For  any  (z,w)  c  *  R^,  a  pair  (x»y)  Is  a  saddle-point  of 
H  iff  (x,y)  t  don  H, 

H(*',y)  «•  Iz^fx*)  <,  H(x,y)  +  Sz^g^xJ.Vx'  c  dorn^H 
and 

H(x,y)  ♦  2*jfj{y)  ♦  5Jwjfj(y') ,  Vy’  t  dem^. 

Now  It  Is  an  easy  exercise  to  show  (using  (7.5)  ar.d  (6.1))  that  for  any  con¬ 
vex  function  f  and  any  convex  set  C  containing  ri(dom  f ) ,  x*  c  3f(x)  Iff 

f(x')  >_  f(x)  ♦  <x\x‘  -  x>,Vx'  c  C, 

But  r1(dom(H(x,* )  ♦  Zw^fj))  ■  rUdom^H)  and  r1(con»(h(  •  ,y)  r  *• 

rl(doaij)  whenever  (z,w)  *  >  R?  and  (x,y)  c  dwn  H.  Hence  it  follows 
from  these  facts  and  (23.8)  that  for  (z,w)  c  r£  *  Rj,  (x,y)  is  a  saddle- 
point  of  H  Iff  (x.y)  c  dao  H, 

i  fW 

o  c  a^'x.y)  -*  !Ca(z1g1  )(x) 

end 

o  c  3pH(xvy)  ♦  ^3(Wjfj)(y). 

The  lefiina  fellows  trivially  from  this  by  Corollary  7.2.1  and  the  definition 
of  S  . 

2,  W 

THEOREM  7.5.  The  set  of  stable  optimal  solutions  of  S( K)  Is  precisely 

IS  J(i.w)  c  RP  *  Rq), 

and  when  S(K)  j_s_  strongly  consistent  this  set  coincides  wl th  the  set  of  al  1 
optimal  solutions  of  S(K).  If  S  t  p,  then  (z.w)  Is  a  Kuhn-Tucker  vec- 
tor  for  S(K) ;  the  converse  holds  whsr_  the  program  dual  to  S { K )  js  strongly 
consistent. 

PROOF.  The  assertions  fellow  immediately  from  Cemnta  7.4  and  Corollaries 


is  o . 1 7.2  and  6.5.1. 


5' 


This  means  that  In  general  ther»  no:  exist  <i  gnoa  l.oor.ingt  nw  1 1 1 

plier  principle  for  ordinary  saddle  pr>r»ren;;.  In  certain  cl  rcun  stances , 
though,  there  Is  an  analogue  of  (28.1). 

COROLLARf  7.5.1.  Assume  tjiat  the  jjrojrTjffl  duaj  to  $(K)  j_s  «■  tr<. ng ly 
co»j_sj_s_tent.  l_f  S(K)  has  a  unique  >  unre-r  ucler  vector  ( i  ,w)  ( or  PCjUlva  ’  enc  • 

J_y,  a  unique  optima)  solution  «,  ^  Its  dual  problem) .  then  the  set  of 

staple  optimal  solutions  or  S(K)  i  nonempty  a_nd  equals 

7ne  next  result  characterizes  dual  strong  consistency  for  ordinary  sad¬ 
dle  programs. 


LFMMA  7.6.  The  projjram  dual  to  S { .< >  jj  strongly  consistent  j_M  the 
two  sets 

ret  concur  Tire:  cone  g,fi...fNrec  cone  g^, 
rec  cone^Mn*^’;  rme  t^O  .  .Orec  cone  f(? 
roe  upped  under  seder  multlpl ’caup.i  by  -•  1 . 

PROOF.  By  lemma  6.4  it  suK'ces  to  sha^  that 

z 


(re c.K)(x)  >o  Itf  r.  i.  rec  :or.e,H r\  ('  \  rec  n re  g 

1 


ar.d 


-1 

(rec^K)(y)  '  o  iff  y  r  re:  cone^H  ,'i  ,r"\  rec  cone  f.. 

^  J-l  J 

Lot.  HC,...,H  be  as  in  the  proof  of  Ineo-om  ?.l  and  l^t  (u,x)  c  rl  t;. 

By  (9.3), 


rec  K(u,x,*  ,♦)  t£rec  M.  (u,x,- . 


Observe  that  trivially 


rec  Hc ( u , x , • ,  • ) '  v ,  / )  •  'Ou  {( x .  - }  (y) 


^nd 


re:  H, (u.x,-  ,• )( v.y)  c  fnr  i  -•  i,..  ,p. 
» 

With  :r.e  a  id  of  (3.5)  and  (9.1)  It  1i  ca-  y  to  -.now  that 
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rfcC  Mp4j^u*A*  ,y>  ’  ^(.V*v;)  !^PI{  »«»-■  ft)) 

for  J  *  I ...  -  .q.  These  fac'.:-,  together  imply  that  f  rec^X);  v,y)  /*qc..ils 
( rvc^H)  (y)  the  r.  ( rec  1^)(y)  <  v  for  j  -  n  nd  equals  other- 

vile.  This  establishes  the  second  equivalence  stated  above.  The  f'ist  can 
oe  proved  s<n; larly. 

Theorem  ,'.5  and  Corollary  >'.*».  I  are  actually  valid  for  any  general  izoj 
saddle  o rogr^i  >  provider  th(»t  for  each  (z,w>  r  kK  *  fig  the  set  *  w  is 
deMnejl  to  be  the  set  of  na1~;  \  ,j  such  that  (x,y)  and  (z,w)  satisfy 

l*>c  extremal  1  ry  conditions.  Tv  proofs  <50  through  erectly  the  sare  except 
that  the  definition  of  S,  j.!;jy:.  the  ro’e  of  l.eima  7.1. 


i  idle  Programs  h eriche I  T y pc 


Throughout  t*  :•  section  t.  is  *•  closed  proper  ron*..,v«-<.ofive/  fuucton 
on  fi01  »  RM ,  L  Is  a  closed  proper  convex-ccncave  function  on  v?  »  i)C\  and 
A  a  Is  a  linear  transformation  ft err  R111  <  R*’  to  R1  *  R°.  L v.’flne 

y.  l;  (x  «.  doirvKj  A.  x  (.  dixo.,:}. 


and  consider 
(1) 
(I!) 

Wnen  m 


>  T  <y  l  do.%<iA.>y  t.  donipl  ]  , 
l  -  (2  l  dom,L*jA|2  c  lUn^K*}, 

W  ‘  (  i  i  <lam^l*jA$w  c  dom^K*} , 
the  lollowing  pair  cf  minimax  problems. 

Find  the "saddle- points  of  K  -  !A  with  respect  to  X  *  Y; 

Find  the  saddle-po : r*s  of  L*  -  K*A*  with  respect  to  7  *  W. 

•=  p,  n  q,  a  is  the  identity  transformation  and  !  is  given  by 
o  If  x  *:  P*’1  end  >  t  Kf 
L(x,y)  a  ^  If  *  /  and  y  t  R? 

"--if  y  <  r';. 


problems  (I)  and  (II)  are  those  considered  by  Rock  stellar  in  [431.  TPe  re  - 
srltr.  in  I..ebedev-Tynjanslci [30],  rynjansxii  [53],  and  some  of  the  results  in 
Tynjanskri  [57]  are  improved  In  this  section. 

Define  a  functio*’  ♦  or)  ( Rr  *  Rm)  *  (R^  >  Rn)  by 


^  F( x.y)  -  L(u  +  A,x,v  f  A ,j) 

( 

if 

( u ,  x )  r  r 

and  (v.y)  <  .' 

*(u,x,v,y)  -  •{  r- 

i  t 

tij.x)  c  r 

and  (v.y)  ./  a 

1 

**  — u» 

if 

(u,x)  (/  r 

where 

r  -  1  (u,x)jx  t  CoirjK,  u  ♦  A j  x  aoWjl), 

A  -»  {{v.y'jy  c  dorr/,  v  »  /o,y  c  dofn.,1 } . 

C.  K. 

UMMA  8.1.  The  function  t  is  clo  .».  i  proper  concnve-convex  wltn  demajn 


T  *  t. ,  and 


1 00 

r i  !'  e.  r  1  ( <1  xn  /),  o  '  t  ;■  * ( don ^ *. }  / , 

ri  £  -  !(-.  ,y'[y  c  ri(  i>n.?K ) ,  v  +  r.  1 1  (  dou  , . ' ' , 

PROOF.  trivially.  r  is  convex.  Define  a  {u|(b.v.i  t  D  for  each 
> .  Then  r,  is  empty  wN»n  x  t.  d:x.i,K  and  f-auafs  dom.L  -  A,  x  when 

f  >:  domA.  Henct  (6.8)  irapile-  that  i-j.;;;  L  ri  r  iff  >  «•  ri-r  . .ul 

u  <.  r((dom,L  -  A,x).  This  establishes  the  voirvj’a  for  n  .  ,  a  ;d  r>  ■'  or m  r u r 
ri  a  is  similar.  From  these  tannuiss  ana  the  fact  ’hat  K  and  L  are  tlnse 

and  proper,  It  is  not  hard  to  verify  (using  (31.3))  that  $  has  the  (iropen.it* 

asserted. 

By  Lenma  b.1,  $  detennnes  a  generalized  saddle  program  _•(*)  cn 
R*n  x  Rn  -v f th  perturbations  In  *  kq.  (he  formulas  for  ri  r  and  ri  \ 
<mply  that  3 (<> )  is  strorg’y  consistent  iff  A  n(dom  K)  meets  ^dom  L). 
and  in  this  case  Theorems  2.3,  4.2  and  6.2  Imply  that  [k  -  LA]  is  well-de¬ 
fined  and  gives  the  primal  problem  associated  with  S(«).  Sy  uns 

she  same  as  (I).  A  generalized  saddle  program  having  the  form  of  F  «)  is 
said  to  be  trf  FencheT  tvj)e . 


It  can  be  computer.  as  an  exercise  that  th" 

program  S(v) 

dual 

■.r  b(*) 

ray  be  given  by 

^-L*(z,w)  -  <*’,s  ;  A tz>  t  f  A 

If  (  S  -  L )  *.  '1 

and 

i.  ,. 

f  (  S  » 2 ,  t  ,v.')  31  A  — 

!f  (s.z)  l  ; 

.'.rid 

, 1  -0 

U„ 

If 

,  '  t  f  *• 

where 

ri  =  {(£,z)lz.  e  dom.L*,  s  a  A'z  t  dom^K*}  , 
ii  ( ( t  ,w) ;  v  c  dom.  i,*;  t  a  t.  dom,/*:  . 

L  IL  c 

Thus,  the  dual  of  a  program  of  f^nchel  type  is  another  program  cf  Fe.ichei 
type.  Hence  3(?)  is  s^ongly  consistent  i'f  A':ri(Hom  '.*)  meets  *  (oor  \* 
and  in  this  case  ft*  -  K*A*]  is  well-  Jef ;  red  and  gives  the  or 'me:  problem 
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associated  with  S(t)  (l.e.  the  dual  problem  associated  with  S ( ♦) ) .  This 
problem  Is  the  same  as  (II). 

With  these  facts  In  mind.  It  Is  clear  that  all  the  results  of  56  can  be 
translated  Into  assertions  about  problems  (I)  and  (II).  In  the  remainder  of 
this  section  we  Illustrate  some  of  this. 

A  saddle-point  of  X  -  LA  with  respect  to  X  *  Y  Is  called  an  optimal 
solution  of  (I).  U  li.  convenient  to  say  an  optimal  solution  of  (I)  Is  stable 
iff  It  Is  a  stable  optimal  solution  of  S(t).  Similar  definitions  are  used 
for  (II). 

LEMMA  8.2.  If  A  r1(dom  K)A  r1(dom  L)  i  4,  then 

sup  Inf  K  -  LA  sup  Inf  L*  -  K*A*  <  Inf  sup  L*  -  K*A*  <_  inf  sup  K  -  LA. 

X  Y  W  1  Z  W  Y  X 

PROOF.  By  the  dual  version  of  Corollary  6.5.2. 

LEMMA  8.3.  In  order  that  A*r1(dom  L*)r\r1(dom  X*)  f  d,  It  is  necessary 
and  sufficient  that 

(reCjKHx)  >_  (rec^LMA^x)  Imply  (rec^K)(-x)  >_  (rec^L) ( -A^x) 

and 

(rec2K)(y)  <  (rec2L)(A?y)  Imply  (rec2<)(-y)  <  (rec^H-Agy). 

PROOF.  The  lemma  will  follow  from  Lemma  6.4,  once  It  is  verified  that 
(reCjO(o.x)  >  o  Iff  (rec^K)(x)  _>  (reCjL)(Aj.x)  and  (re c2*)(o,y)  <  o  Iff 
(rec2K)(y)  ^  (reCgLHA,^) .  Only  the  second  equivalence  will  be  checked,  as 
the  first  is  analogous.  For  each  (u,x)  c  rl  r,  It  fellows  from  Lemma  8.1, 
(9.3)  and  (9.5)  that 

rec  ♦(u,x,*,*)(v,y)  -  rec  K(x,*)(y)  -  rec  L(u  +  AjX,-)(v  «■  A^). 

Hence,  (rec2*)(o,y)  _<  o  Iff  rec  K( x , • ) (y)  <.  rec  L(u  +  AjX.-HA^)  holds  for 
each  (u,x)  c  rl  r.  But  this  latter  condition  occurs  Iff  rec  K(x,*)(y)  £ 
rec  L(u, * ) (A^y)  holds  for  each  x  c  rlfdom^K)  and  u  c  rifdon^L),  which 
occurs  iff  (rec2X)(y)  <  (re c^L) ( A^) . 


JO? 

It.  can  be  shown  that  the  least  member  of  the  lagrvrtylflp  f  $,'«)  is 
the  function 

K(x,y)  t  L/fz.w)  -  'A^x,z>  -  <A.^v,v>  If  (x,y)  C  and  (2,w)  t  cl  0 

(x.w.y.z}  -*•  +«  If  (x.y)  c  C  vi.,  f  cl  a 

*■  -  if  I x*y)  f  C 

where  C  *  D  *  (dom^K  *  dom^c’-)  *  (dofn^K  *  dotr..,!.*’)  is  the  domain  of  .ho 
jranglan.  Froai  this  it  follo'ws  easily  by  (  35.1),  (  36.1)  sod  (37.4)  tha'i  two 
pairs  ( x , y )  c  if  *  fi"  and  {  7,w)  e  *  Rl*  satisfy  the  extremal  i  fy  condi¬ 
tions  associated  with  S(*)  and  $(?)  Iff 

A(x,y)  t  dl*(-c,w)  and  A*(z,w)  z  3K(x,y). 

The  next  three  results  are  simply  translation;,  of  Theorem  6  17,  CoroT  1  -.ry 
5.17.1  and  Ccroliary  5.17,2. 

THEOREM  8.4.  A  pair  (x,y)  k  o_  _sja_b1e  o£tjh?«i  ‘'.o'. 'ill Pi1  2l  1  *'  If* 
there  exists  a  pal r  (z,w)  such  that 

A(x,y)  c  aLf,(z  w)  and  A*(z,w)  •_  3K(x,y)  (") 

Uually,  a  pair  (z.w)  k  a  s  ‘  a  Ik  oot  imal  solut  icr:  o£  (II)  jiff  there  exists 
a_  £3lr  (x,y)  such  that  (M  h.pJill- 

THE3REM  8.5.  Problem  (?)  has_  j_  stable  optima1,  solution  kf  Problem  (II) 
does ,  jjn  which  case  the  optimal  «akjes_  ip.  (I)  ino  (IT)  arc  ecjU'd) 

THEOREM  6.6.  i_f  A  ri(ho»T-  K) C\  ri(dc»Ti  I.)  /  <J,  then  avi  m  Oi>tin,ci  solu¬ 
tion  of  (I)  is  stable.  Dually ,  if  A*ri ( dem  ik'T'.  ri(dom  K  *)  ;■  t ,  ‘h  ever/ 
optimal  solution  of  (II)  j_s  stable. 

To  go  along  with  these  results,  we  heve  two  existence  results.  The 
is  a  corollary  to  the  next  theorem. 

THEOREM  8.7.  Assume  A  rj(dom  K)flri(cJom  L)  f  jJ.  Then 
dom(K  -  LA) *  C.  dem  K*  -  A*dom  l*, 


and 


rt  (doiri  K* 


10  J 

-  A*Oom  I.4)  c  :om'v ►  ) 4 

ur 

rec.(K  -  LA)  *  r°c,K  -  (rr..l)A 
■j  J  J  J 

or  J  *  1  and  2 . 

°KDCF.  The  >r\t  > dc I u'  i  or  to’ lows  t r">n  Corolla*  -.e ;  4  S.j  jn<J  1  4.?. 

"  .V.1  second  assertion  follows  fro»«  If  ',)rt*ps  4.H  :\wi  Z.f,  wi  th  top  help  of  (b  1.1' 
i.OVi  L  CAR'*  8.7  !.  As  sc  rip  A  nf.'ioir,  f  i  A  .  '  -v  t.  /•  y  , 

A4  -ti  j.'xn  i  *)  r\  rl ( >io.i  h")  r  o,  ^  j 

r  '“iC  .X  (  t  e  *  .1. )  A . 

J  .)  J  J 

jor  1  1  a nrf  2.  I_Ly'i  there  exjis^s  or.  opt.  nal_  sojjt. i  ;r  ov  (I). 

PFOOP.  The  theorem  ana  (6.3.1)  iinclv  that  n(oo"i(h  -  equals 

r  t  ( rJom  K*  -  A*dom  L*; ,  which  equals  rl-.O’vr  /M  A*ri.uom  L*;  bo  (b  ^  on<'« 

(6.6.2).  Hence  (o,i>)  •.  n  (awn!  K  :  *'♦)  ;  •  •'  •  r-i  ( dcm  L  * )  f\ r  (  Com  •  * )  r  t. 

l(;us,  (37.5.3)  implies  there  exists  a  s.tcidl--pj.r't  of  fY  -  .  AJ. 

A  more  general  existence  r-*s ■  •' t  Is  the  f»:l  ’  jwlnsj. 

'FcL-PLM  8.8.  Assume  fhat  A  r  i  ( com  -^dom  !.;  /  P  ard  f.i-at  fne 
To  1 1  ow  i . ,q  two  rondi  t  iprs  are  satisfied 

;  .  rec  mx,')(y)  <  roc  l(  ,• )  (A.,y)  for  every  x  ..  ri  <. 

then  'ec  K(x,-i\-y)  y  r«*c  n  A  jX  ,• )(  -A^y)  for  e  s  e  r  y  x  .  ri  X. 

t'b)  If  rec  K(*,y)(x)  _  roc  L{  •  ,A.  y  •  (A ,  x;  every  y  ».  rj  Y,  Mien 

rec  K(*,y)(-x)  y  rec  l(*  .A^yH-Ajx)  toy  every  y  <  n 
Then  there  exists  a_o  optimal  so.  ut  ion  of  ( 1 ) . 

IROOr.  Since  [$3J  -  {.*  -  ^A),  the  thpr  rem  w;  li  follow  iiwicdi  ately  trrvr 
Corollary  6.2.1  cnce  it  is  checked  that  reo^(K  -  m)(y)  <_  o  Iff 
rec  k(>,  }(y)  rec  L(A  ^x,-)(A^/)  f  eve*-;  <  ,  r<  X,  ana  shat 

rec,(K  -  LA)f.v)  y  0  iff  rec  K(-,y)(v)  '•  r^'  V  •  /••,>  /  '  A,.<)  for  every 

y  i  rl  Y.  We  Show  only  the  first  tools.  rn;r,  ar  f'.p  second  1*.  sinilar.  6; 


’  llr-  I  .  !  I  L  I 


K\ 


lhicrev  4.?  and  PH, 

( s  -  :  Al 

has  domain 

X  -  Y 

and  ontai 

d  9lvr-n  by 

,<(x.y)  - 

LA(x,y)  if 

X  L  X 

and  y  r.  Y 

U*.i;  v 

1 

P 

If 

x  c  X 

and  y  /  Y 

] 

\* 

**  r  Y. 

rrac  thP  toother  w  Pi'  \  ':> .  J )  and  (<»  b) ,  it  follows  that 
roc  -  *<r  r. (  x . - ) i  y)  -  n  c  i(A,x,*)(Ay) 

for  over/  /.  <.  rt  x.  ;.1«  <  i-?‘yx  •  *.rtl(>)  »  'mjd  trec  H(x  .•  Py) !  *  .  •*!  X), 
cr,o  oquivaun  e  '  o  <  .  ovr>. 

Finally,  we  re-mark  that  coudi ti  »ns  (a)  and  (h)  ai:ove  art*  satisfied  for 
axaitplc  when  A  *Y  is  oounden 


APPENOI X:  Polyhedral  Refinements 


Polyhedral  saddle- functions  have  much  nicer  properties  then  do  arblfrao 
closed  saddle- functions.  Consequently!  many  of  the  results  In  the  thesis 
adult  refinements  when  some  of  the  saddle-functions  involved  are  polyhed-ai. 
These  refinements  generally  Involve  weakening  the  hypotheses  In  either  o.' 
both  of  two  ways.  The  first  way  may  be  described  loosely  as  follows:  if  * 
theorem  can  be  proved  using  a  hypothesis  of  the  form  (C  *  D)  r\  rlfdom  K)  f  A, 
where  K  Is  a  closed  saddle-function  and  C  and  0  are  convex  sets,  then 
the  same  conclusions  (and  sometimes  even  sharper  ones)  can  be  obtained  from 
the  weaker  hypothesis  (C  *  D)H  dent  K  f  0  when  K  Is  actually  polyhedral. 
Refinements  of  this  type  rest  ultimately  on  the  fact  that  the  main  tools  frc«n 
convex  function  theory  which  are  used  In  the  proofs  (e.g.  (16.3),  (16.4), 
(23.8)  and  (23.9))  admit  polyhedral  refinements  of  the  same  type.  This  covers 
most  or  the  polyhedral  refinements.  However  the  results  which  are  essentially 
assertions  about  the  existence  of  saddle-points  a&dt  refinements  of  a  differ¬ 
ent  sort.  Generally  speaking,  such  results  hypotheslie  conditions  of  tha  form 
(recjk)(-x)  ^  6  whenever  (recjK)(x)  o 
and 

(reCgKM-y)  <  o  whenever  (rec^K)(y)  ±  o. 

These  are  dual  to  the  condition  (o,o)  c  r1(dom  K*)  and  hence  Imply  the  exis¬ 
tence  of  a  saddle-point  of  K.  However,  when  K  Is  proper  and  polyhedral, 

It  can  be  shown  (using  (23.10))  that  don  3K*  »  dom  K*  and  henre  that  K  has 
a  saddle-point  Iff  (o,o)  e  dom  K*.  Thus,  for  K  proper  and  polyhedral,  only 
the  weaker  conditions  of  the  form 

(rec-|K)(x)  <_  o  for  all  x 
and 

(rec^K/fy)  _>_  o  for  all  y 


are  needed. 
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eerier  tj  1 

>1  ’ioci.e. 

,  •  -1  1 

,  there  :* 
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i  rr/’t.g 
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trjof-.  j* 

.1  lyh 

T_.  ;eJ. 

«1  vers 1 

op  <>f  • 
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yVu  li  .vr  i  i 
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clo.-ei  p* 

op':''  re, 

::  v?-:.c  r:  •  •  : 

1 1  j : . . : ;  v.)<i 

v.er  ?; 

•//('  )  i  V 

r  b) 

h  '-C  ^  \  >■ 

■ 

r  hi 

•  !  i  dc.r  <) 

C.  C  r 

L-  Cl  M.'lp  < 

i\  \Siird 

I  r  merits 

in  r*’t-  tt' 

;  U  '  s  * 

generalization  of  (6,5).  it  Is  \}\ l  ».n>  cony*;  t..  ^.n  f r.  1 

c 

satisfying  Cj  H  rl  Z.,  t  jJ,  ere  tos  ' .  *  C-(  C ,}  C  C,  /'.r!  C-.  «r- 
'j  r\  cl  Cj  C  cl(C,n  C^).  This  cir,  be  prv/c-i  hy  first  os' -.5  r  ■.••:. -..r  \i  u<<'. 
argument  based  on  (11-3)  to  »ho*  that  r\  rt  r!  C,,  f  t-  r-ta-"*  '  - 
Cj ruff  £2*  and  then  applying  (^.5). 

The  results  in  $Z  *h?;h  u>*  the-  hyrotnesi?  rang*  A ;  i \  b>>;r  '  1  0  c: 

:e  proved  assum''ig  just  that  r;»oge  AT t  Ir..;  ,<  ;•  ;«  when  K  1*  .»«-  lyi-eor j . . 
The  hypotheses  in  Theorem  7.2.  Z.'  ^nn  3  5  .; jr.  0-.  v-,2aher.;J.  00 “  i y 
for  example,  the  .  end  us  ions  of  Jhe<*r«m  3  i  si.  II  hold  :i  K  i.;  r^v  r  pel  / 
hedral  and  (u,v)  r.  t  deyr  K  ort« the  cs  xlitions 


inf  f  rc-r  K(*  ,y)(;<)jy  s.  .brr-.,.\  t.y  v}  0  wheervr- 

1  c 


*  r':V  ;  C 


sup  { rec  K(x,«  ) (.*/)  j x  t  dttr.jK  .  r,x  ■  u)  0  ..hhi.ev-':  r.j--  >••  0 
The  polyhedral  rnf1new.;tr.  rot  ,  •"■  b.-n  *j  y.^ri  r-*vir-:  >nr-.  fj»  vl  vh-i 
all  of  the  saddle- functions  K-, - ,K  are  po»jhee.-?l.  i-,  this  ..-.so  .ho 
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hypothesis  (*)  can  be  replaced  by 

don  K^Pp.  .Hdoei  f  i. 

In  the  nixed  case,  when  for  example  the  saddle-functions  Kj,...,Kr  are 
polyhedral  but  Vr-  ,K$  are  riot,  we  can  still  prove  everything  with  (v‘) 
replaced  by 

dom  K^n  . .  .Pldcm  Kf  O  r1(dom  ^  )H  . .  .Plr1{dom  Kj  f  d. 

The  proofs,  however,  do  not  follow  from  fi2  and  3  by  the  device  of  represent¬ 
ing  [ ♦...*  Ksj  as  [XA]  { cf .  Theorem  4.6).  Instead,  one  must  carry  out 
proofs  parallel  to  those  In  u2  and  3  but  which  appeal  to  (20.1)  in  place  of 
(16.3)  and  the  polyhedral  version  of  (23.8)  in  place  of  (23.9). 

Concerning  j6,  define  a  generalized  saddle  program  S(K)  to  he  polyhedral 
Iff  K  Is  polyhedral.  It  is  easy  to  see  that  5(K)  Is  polyhedral  iff  its 
dual  program  S(L)  Is  polyhedral.  The  polyhedral  refinements  of  the  first 
type  described  above  take  the  form  of  reDlacing  the  hypothesis  "S(K)  is 
strongly  consistent"  by  the  hypothesis  "S(K)  is  polyhedral  and  consistent." 

The  refinement  of  Corollary  6.2. 1  combines  both  general  types  of  refinement: 

If  S(K)  Is  polyhedral  and  consistent,  then  it  has  an  optimal  solution  iff 
(rec^Ko)(x)  _<  o  for  all  x  and  trec^KoMy)  ^  o  for  all  y. 

The  polyhedral  refinements  for  s6  yield  refinements  for  s 7  when  all  of 
the  functions  H,  fj....,fq  are  polyhedral  and  also  for  $8  when 

both  K  and  L  are  polyhedral.  The  mixed  case  of  §7  does  not  appear  to  go 
through  in  general.  The  troublesome  spot  is  establishing  a  version  of  Corol¬ 
lary  6.17.2,  i.e.  establishing  the  existence  of  Lagrange  multipliers.  The 
mixed  case  of  J8,  though,  does  allow  refinements  of  all  the  results.  The  hypo¬ 
thesis  "S(a)  is  strongly  consistent,'  I.e.  ri(dom  L) H  A  r1(dom  K)  t  t,  is 
weakened  by  replacing  ri(dom  X)  with  dom  K  in  the  event  K  is  polyhedral 
and  by  replacing  ri(dom  L)  with  dom  L  in  the  event  L  is  polyhedral.  Then 


I 
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everythin?  goes  through  by  appealing  to  the  polyhedral  case  of  i 2  and  the 
nixed  case  of  f4. 
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